RACE: Scalable Statistical Estimation of Functional Consistency in LLM Neurons¶
Conference: EMNLP 2026
Paper: ACL Anthology
Code: https://github.com/Nexround/RACE
Area: LLM Reasoning
Keywords: Mechanistic Interpretability, Neuron Functional Consistency, Residual-Direction Alignment, Bayesian Uncertainty Estimation, Targeted Suppression
TL;DR¶
Addressing the excessive computational cost and sample-to-sample variance oversimplification in existing mechanistic interpretability, this paper introduces RACE, a forward-pass statistical framework that decomposes module residual updates and leverages Normal-Inverse-Gamma conjugate Bayesian modeling to rank domain-functional neurons with extreme scalability and high intervention specificity.
Background & Motivation¶
Large Language Models exhibit impressive capabilities across code generation, mathematical reasoning, and multi-turn planning, yet deciphering their internal mechanisms remains a fundamental challenge in mechanistic interpretability. Prevailing techniques—ranging from causal tracing and activation patching to gradient-based attribution and dictionary learning via sparse autoencoders—predominantly focus on instance-level analyses. While highly capable of explaining model decisions on isolated inputs, real-world deployment needs, such as domain-specific model pruning, safety capability auditing, and fine-grained behavioral steering, demand population-level characterizations of neuron functions across an entire target domain.
Existing task-neuron mapping methodologies face a severe dual bottleneck: computational inefficiency and statistical oversimplification. Gradient-based attribution and iterative causal interventions require backpropagation across thousands of tokens, incurring prohibitive computational overhead that severely hinders domain-wide neuron auditing on 10B+ scale models. Concurrently, prior approaches typically aggregate neuron contributions across observations via simple empirical averaging. This scalar averaging discards cross-sample distributional dynamics, inadvertently rewarding bursty "outlier neurons" that fire violently on a few idiosyncratic prompts but remain silent or antagonistic across the broader target domain, thereby polluting domain-specific rankings.
Because Transformer computation is mediated by incremental additive writes to the residual stream, neurons serving consistent domain-specific functions must systematically contribute positive, low-variance alignment along the host module's update trajectory during the forward pass. The core idea is to evaluate neuron functional consistency at forward-pass cost via Residual-Direction Alignment (RDA) onto the host module's update axis, distill noisy per-observation signals into posterior mean and variance distributions through a Normal-Inverse-Gamma (NIG) conjugate model, and apply Conservative Alignment Magnitude (CAM) lower credible bounds alongside Reference-Set Filtering (RSF) to isolate high-fidelity, domain-specific neurons.
Method¶
Overall Architecture¶
The RACE framework operates in four cohesive stages: observation collection across target and reference sets, module-local evidence extraction via Residual-Direction Alignment (RDA), closed-form Bayesian evidence aggregation using an NIG conjugate model, and candidate selection governed by Conservative Alignment Magnitude (CAM) combined with Reference-Set Filtering (RSF). Operating strictly during standard forward passes, the pipeline maintains \(O(1)\) streaming sufficient statistics per neuron, computes analytic Student's t marginal posteriors, penalizes noisy or sparse observations through a conservative credible lower bound, and filters out general-purpose linguistic infrastructure before downstream intervention.
%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
A["Target & Reference Observation Sets<br/>D_c and D_ref with active token positions T_c"] --> B["Residual-Direction Alignment RDA<br/>Module-local residual decomposition and unit projection"]
B --> C["Bayesian Conjugate Evidence Aggregation<br/>Online sufficient statistics and Student's t marginal posterior"]
C --> D["Conservative Alignment Magnitude CAM Scoring<br/>One-sided credible lower bound penalizing variance & noise"]
D --> E["Reference-Set Filtering RSF<br/>Excluding shared general neurons via reference ranking"]
E --> F["Isolated Domain-Specific Neurons<br/>Suppression intervention & fine-grained behavioral steering"]
Key Designs¶
1. Residual-Direction Alignment: Forward-Only Geometric Attribution to Local Residual Updates
Aligning intermediate neurons directly against the final token logits conflates local functional contributions with network depth and downstream non-linear transformations, as intermediate representations are frequently rotated, overwritten, or erased by subsequent layers. RDA addresses this by evaluating each neuron against its host module's own residual update \(\Delta r_{l,m}\). The module output is decomposed into an exact sum of rank-1 neuron writes: $\(\Delta r_{\mathrm{ATTN}} = \sum_i h_i W_O[:, i], \qquad \Delta r_{\mathrm{MLP}} = \sum_j a_j W_{\mathrm{down}}[:, j]\)$ Normalizing this update defines the observation-specific unit axis \(\hat{d} = \Delta r / (\|\Delta r\|_2 + \epsilon)\). Projecting each neuron's vector write \(v_j\) onto \(\hat{d}\) produces the activation-modulated signed alignment score: $\(e_{\mathrm{MLP}, j}^{(t)} = a_j^{(t)} \cdot (W_{\mathrm{down}}[:, j]^\top \hat{d}_{\mathrm{MLP}}^{(t)})\)$ Geometrically, \(v_j = e_j \hat{d} + q_j\) with \(\sum_j q_j = 0\) and \(\sum_j e_j = \|\Delta r\|_2\), establishing \(e_j\) as an exact partition of the module update norm and its first-order sensitivity under neuron rescaling. This projection filters out orthogonal components that cancel under superposition, yielding signed, module-local evidence at zero backpropagation cost.
2. Bayesian Conjugate Evidence Aggregation: Closed-Form Modeling of Evidence Dispersion
To distinguish stable domain-wide contributors from spurious outlier spikes, RACE treats per-observation evidence \(e_j^{(t)}\) as noisy draws from a latent Gaussian \(\mathcal{N}(\mu_j, \sigma_j^2)\) governed by a conjugate Normal-Inverse-Gamma (NIG) prior \(\mathrm{NIG}(\mu_0, \lambda_0, \alpha_0, \beta_0)\). Given \(n\) observations across \(T_c\), the model tracks the empirical mean \(\bar{e}_j\) and centered sum of squares \(\mathrm{SS}_j = \sum_t (e_j^{(t)} - \bar{e}_j)^2\), updating hyperparameters via closed-form arithmetic: $\(\lambda_n = \lambda_0 + n, \quad \alpha_n = \alpha_0 + \frac{n}{2}, \quad \mu_{n,j} = \frac{\lambda_0 \mu_0 + n \bar{e}_j}{\lambda_n}, \quad \beta_{n,j} = \beta_0 + \frac{\mathrm{SS}_j}{2} + \frac{\lambda_0 n (\bar{e}_j - \mu_0)^2}{2 \lambda_n}\)$ Integrating out \(\sigma_j^2\) yields the marginal posterior over mean alignment as a Student's t-distribution with \(2\alpha_n\) degrees of freedom and scale parameter \(\sigma_{\mu, j} = \sqrt{\beta_{n,j} / (\alpha_n \lambda_n)}\). The scale parameter directly reflects epistemic and observation uncertainty, widening intervals whenever evidence is scarce, highly dispersed, or fluctuating in sign.
3. Conservative Alignment Magnitude: Quantile-Calibrated Lower Credible Bound Ranking
RACE scores neurons using the Conservative Alignment Magnitude (CAM), defined as the one-sided \((1 - \gamma)\) lower credible bound on the posterior mean of positive module alignment: $\(\rho_j(\gamma) = \mu_{n,j} - t_{1-\gamma, 2\alpha_n} \cdot \sigma_{\mu,j}, \qquad \mathrm{CAM}_j(\gamma) = \mathbb{I}[\mu_{n,j} > 0] \max(0, \rho_j(\gamma))\)$ where \(t_{1-\gamma, 2\alpha_n}\) is the \((1 - \gamma)\)-quantile of the Student's t-distribution. Neurons that achieve high average activation through a handful of massive outliers suffer an inflated \(\mathrm{SS}_j\), which widens \(\sigma_{\mu, j}\) and drags the conservative bound down to zero. Conversely, neurons with consistently positive, low-variance alignment across the observation set maintain tight confidence radii, earning top rankings while noisy, sign-fluctuating candidates are rigorously penalized.
4. Reference-Set Filtering: Disentangling Domain Specialization from Shared Linguistic Infrastructure
Because foundational language models represent features in superposition, unconstrained domain rankings can be dominated by polysemantic or general-purpose neurons supporting fundamental language generation. RSF resolves this by scoring neurons over a general reference corpus \(D_{\mathrm{ref}}\) (e.g., WikiText-2 or FineWeb) under the identical scoring rule, identifying an exclusion set \(B_{\ell}^{\mathrm{ref}} = \mathrm{Top}_{K_{\mathrm{ref}}}^{\mathrm{sref}}(U_\ell)\) for each layer and module. When selecting candidate neurons from the target ranking \(\Pi_\ell^{\mathrm{tar}}\), RSF traverses the list, skips any neuron present in \(B_\ell^{\mathrm{ref}}\), and extracts the first \(K_{\mathrm{sel}}\) domain-exclusive neurons. This ensures that targeted suppression selectively degrades the target domain while preserving general linguistic competence.
Key Experimental Results¶
Main Results¶
RACE was evaluated across Qwen3-4B-it, OLMo-3.1-32B-it, and Llama-3.1-8B-it spanning Code Generation (MBPP+, HumanEval+), Mathematical Reasoning (MATH-500, AMC), and general retention benchmarks (MMLU-Redux, GPQA, ARC). Evaluation uses the Intervention Specificity Index (ISI): $\(\mathrm{ISI} = \max\left(0, \log \frac{\Delta_T}{\Delta_G + \delta}\right)\)$ measuring target degradation \(\Delta_T\) against general degradation \(\Delta_G\) with smoothing \(\delta = 0.01\).
The table below reports post-suppression benchmark accuracy (%) and ISI on Qwen3-4B-it in the code domain under \(R_{\mathrm{MBPP+} \setminus \mathrm{WikiText-2}}\) with top-1% neuron suppression per layer:
| Module | Method | MBPP+ (♠) ↓ | HumanEval+ (♡) ↓ | MATH-500 | AMC | MMLU-Redux | GPQA | ISI ↑ |
|---|---|---|---|---|---|---|---|---|
| ATTN | GxAct | 8.73 | 7.93 | 17.20 | 2.24 | 47.67 | 21.72 | 0.26 |
| ATTN | AttnLRP | 81.75 | 86.59 | 90.03 | 75.37 | 80.67 | 46.97 | 0.05 |
| ATTN | Act. Mean | 0.00 | 0.00 | 2.60 | 0.75 | 6.07 | 5.05 | 0.00 |
| ATTN | Emp. Mean | 66.40 | 78.66 | 88.34 | 63.43 | 78.54 | 40.40 | 0.00 |
| ATTN | Emp. SNR | 68.52 | 84.15 | 93.44 | 76.86 | 79.49 | 40.40 | 0.09 |
| ATTN | Neg. CAM | 81.75 | 82.93 | 92.61 | 83.58 | 81.53 | 45.45 | 0.00 |
| ATTN | RACE (CAM) | 65.87 | 80.49 | 90.30 | 68.66 | 79.07 | 44.41 | 0.29 |
| MLP | GxAct | 23.81 | 19.51 | 79.23 | 46.27 | 78.49 | 40.40 | 1.29 |
| MLP | AttnLRP | 0.00 | 3.05 | 61.37 | 35.07 | 76.39 | 31.82 | 1.07 |
| MLP | Act. Mean | 11.38 | 12.21 | 88.73 | 72.09 | 82.61 | 40.98 | 2.22 |
| MLP | Emp. Mean | 5.82 | 2.44 | 86.44 | 63.43 | 82.30 | 55.05 | 2.26 |
| MLP | Emp. SNR | 83.33 | 84.76 | 93.83 | 69.40 | 78.98 | 38.89 | 0.00 |
| MLP | Neg. CAM | 68.25 | 84.76 | 93.23 | 81.34 | 81.72 | 48.99 | 1.04 |
| MLP | RACE (CAM) | 3.17 | 1.22 | 85.41 | 75.11 | 82.35 | 52.53 | 2.65 |
| Base | Qwen3-4B-it | 82.28 | 83.54 | 94.40 | 87.31 | 81.37 | 45.45 | — |
Computational Profiling & Internal Ablations¶
Profiling was performed on Qwen3-4B-it using PyTorch profiler on a 64-input/16-output window (baseline forward pass = 130.1 GFLOPs), alongside intervention evaluation in the math domain (\(R_{\mathrm{MATH-500} \setminus \mathrm{WikiText-2}}\)):
| Method Category | Evaluated Protocol / Ablation | Extra FLOPs | Relative Forward Overhead | MATH-500 (♠) | AMC (♡) | GPQA (Retained) | ISI ↑ |
|---|---|---|---|---|---|---|---|
| Forward Base | Unmodified Model Inference | — | 1.00× | 94.40 | 87.31 | 45.45 | — |
| Gradient Base | GxAct (Gradient × Activation) | 18.77 TFLOPs | 144.29× | 30.80 | 9.70 | 41.41 | 1.46 |
| Gradient Base | AttnLRP (Relevance Propagation) | 18.82 TFLOPs | 144.62× | 46.00 | 31.34 | 43.43 | 0.78 |
| Ablation | Act. Mean (Unsigned Magnitude) | — | — | 58.40 | 47.76 | 40.91 | 0.75 |
| Ablation | Emp. Mean (No Prior, No Penalty) | — | — | 27.20 | 12.69 | 39.39 | 1.36 |
| Ablation | Emp. SNR (Frequentist Ratio) | — | — | 95.00 | 86.57 | 39.90 | 0.00 |
| Ablation | Neg. CAM (Opposite Direction) | — | — | 95.60 | 85.08 | 44.44 | 0.00 |
| Proposed | RACE (RDA + CAM, MLP) | 81.55 GFLOPs | 0.63× | 29.00 | 11.94 | 40.40 | 1.76 |
Key Findings¶
- Drastic Computational Efficiency Advantage: RDA introduces only 81.55 GFLOPs of extra computation (0.63× forward equivalent), representing a \(>220\times\) reduction in computational overhead compared to gradient baselines GxAct (144.29×) and AttnLRP (144.62×), enabling domain-wide auditing on 32B models in minutes.
- Asymmetric Modularity (MLP vs. Attention): MLP neurons demonstrate high domain exclusivity (all-domain Jaccard overlap across Math, Code, and Language is only 0.085), where suppressing 1% of target MLP neurons drops code accuracy from 82.28% to 3.17% with an ISI of 2.65. Conversely, attention output channels are heavily shared across domains (Jaccard of 0.264, a 3.1× increase), functioning as general-purpose routing hubs that lose domain signal after reference filtering.
- Failure Mode of Frequentist SNR: Emp. SNR completely collapses on MLP interventions (ISI = 0.00) because sparse activations yield near-zero empirical standard deviation in the denominator, causing tiny random noise to be ranked highest. In contrast, the NIG prior regularizes posterior uncertainty (\(\sigma_{\mu, j} > 0\)), ensuring robust conservative screening even in low-sample regimes.
Highlights & Insights¶
- Module-Local Decomposition Bypasses Depth Confounding: By projecting directly onto the immediate module update rather than global logits, RACE eliminates depth bias and accounts for later layers overwriting earlier features.
- Closed-Form Conjugate Streaming with Negligible Memory: Maintaining running sufficient statistics requires only 0.95 MB of additional state for all 497,664 neurons in Qwen3-4B-it, adding no measurable peak GPU memory overhead over standard inference.
- Precise Stylistic Steering Without Catastrophic Forgetting: On PyComp-1K, suppressing just 10 neurons per layer reduces Python comprehension generation by 70.9% on MBPP+ while maintaining 97.73% functional pass@1 accuracy on solutions using comprehensions.
Limitations & Future Work¶
- Linear Residual Projections: RDA projects rank-1 updates linearly onto module axes, which cannot capture complex non-linear feature interactions or distributed polysemantic features requiring non-linear decoding.
- Diminished Selectivity in Attention Modules: Attention output channels exhibit high cross-domain reuse, rendering RSF less effective at isolating pure domain-specific attention neurons without degrading broad linguistic routing.
- Sensitivity to Observation Prompt Diversity: Although CAM is robust to finite sample sizes, highly biased prompt distributions can skew the active token set \(T_c\), necessitating structured prompt diversification in practice.
Related Work & Insights¶
- vs Gradient Attribution (GxAct, AttnLRP): Gradient methods require full backward passes with target token backpropagation, suffering from gradient saturation and high compute; RACE is forward-only, \(>220\times\) faster, and yields higher intervention specificity.
- vs Knowledge Neurons: Knowledge neuron localization relies on heuristic activation thresholds over single prompts; RACE operates on rigorous population-level statistical inference with formal credible lower bounds.
- vs Sparse Autoencoders (SAEs): SAEs demand training separate massive autoencoders at each layer; RACE works natively on the model's raw neuron basis, providing instant zero-training interpretability.
Rating¶
- Novelty: ⭐⭐⭐⭐⭐ Formulates neuron functional consistency as Bayesian posterior inference over forward residual stream projections.
- Experimental Thoroughness: ⭐⭐⭐⭐⭐ Spans 4B to 32B models, code/math domains, distributional perplexity shifts, fine-grained syntax steering, and profiler-verified FLOP benchmarks.
- Writing Quality: ⭐⭐⭐⭐⭐ Clear mathematical formulations, clean geometric derivations, and well-structured comparative tables.
- Value: ⭐⭐⭐⭐⭐ Provides an ultra-scalable, mathematically sound tool for LLM capability auditing, targeted pruning, and behavioral alignment.