Beyond Decodability: Reconstructing Language Model Representations with an Encoding Probe¶
Conference: EMNLP 2026
Paper: ACL Anthology
Area: Interpretability
Keywords: Encoding Probe, Linear Regression, Representation Reconstruction, Probing Interpretability, Feature Collinearity
TL;DR¶
Addressing the core limitations of classic decoding probesโnamely non-comparable cross-feature metrics and vulnerability to feature collinearityโthis paper introduces an Encoding Probe that reverses the probing direction to reconstruct internal model activations using multivariate ridge regression, enabling direct feature contribution quantification on a shared unexplained variance (UV) scale.
Background & Motivation¶
Probing has served as a foundational technique for analyzing the internal representations of deep neural networks and large language models across speech and text. In the classical probing paradigm, referred to as a decoding probe, researchers train auxiliary linear classifiers or regressors to predict specific linguistic or physical attributes (such as part-of-speech tags, syntactic tree depth, phonetic labels, or speaker identities) from the hidden activation vectors of target models. While this approach has demonstrated that models like wav2vec 2.0, HuBERT, and BERT encode rich hierarchical information, a high decoding accuracy only certifies that information is linearly extractable or decodable. It fails to quantify how much that feature actually shapes the underlying geometry of the internal representations. Furthermore, because decoding probes evaluate different tasks across varying label spaces, baseline distributions, and difficulty levels, their performance metrics cannot be compared on a shared scale of representational contribution.
Even more problematic is the pervasive issue of feature correlations in natural language and speech data. For instance, lexical identity and grammatical categories are inherently collinear: many words appear almost exclusively with a single part-of-speech tag. Consequently, a decoding probe achieving high accuracy when predicting grammatical labels may simply be leveraging lexical identity shortcuts rather than true abstract syntactic structure. Similarly, speech waveforms exhibit complex temporal and spectral overlaps between acoustic properties, phonetic sequences, and speaker voice characteristics. Previous attempts to untangle these confoundsโsuch as control tasks, baseline comparisons, or conditional probingโoften require custom task setups and struggle to simultaneously handle multiple high-dimensional continuous and discrete variables.
Drawing inspiration from the brain encoding paradigm in computational neuroscience, this paper reverses the standard probing direction: rather than predicting external attributes from neural activations, it uses interpretable features to reconstruct internal hidden representations. Framing the probing problem as multivariate linear regression naturally unlocks the standard toolset of variance decomposition, allowing researchers to evaluate marginal contributions while rigorously controlling for correlated features. Core idea: reverse the probing direction into an Encoding Probe that reconstructs intermediate neural activations from interpretable features, directly quantifying relative feature contributions on a unified unexplained variance (UV) scale while systematically controlling for feature collinearity.
Method¶
Overall Architecture¶
The encoding probe inverts the conventional discriminative mapping \(f: X \to Y\) (from target representation \(X\) to property \(Y\)) into a generative reconstruction mapping \(g: Y \to X\). The input feature space \(Y\) comprises an extensive ensemble of interpretable features (Full) spanning low-level acoustics, phonetic posteriorgrams, speaker identity one-hot vectors, syntactic descriptors, and static word embeddings. For each transformer layer across targeted speech and text models, a multivariate ridge regression model is fitted to predict the hidden activation vectors. By systematically ablating specific feature subsets from the input ensemble (e.g., Full without Syntax, or Full without Speaker) and measuring the resulting increase in unexplained variance (UV), the framework isolates the unique, non-redundant variance explained by each linguistic or acoustic attribute.
%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
A["Raw Audio Waveform / Text Token Input"] --> B["Multi-dimensional Interpretable Feature Extraction"]
B --> C["Unpooled Frame-level and Subword-level Alignment"]
C --> D["Multivariate Ridge Encoding Regression Reconstruction"]
D --> E["Systematic Feature Ablation and Marginal Contribution Quantification"]
E --> F["Output: Layerwise Unexplained Variance (UV) Gap Analysis"]
Key Designs¶
1. Multi-dimensional Interpretable Feature Extraction: Assembling a Comprehensive Linguistic and Acoustic Feature Suite To rigorously evaluate model representations, the pipeline extracts a broad set of interpretable features representing distinct linguistic and acoustic facets of human language. In the audio domain, low-level acoustic descriptors are extracted using openSMILE at 20ms intervals (the 25-descriptor eGeMAPSv02 set covering pitch, formants, loudness, and MFCCs); phonetic information is captured via 40-dimensional phonetic posteriorgrams (PPGs); and speaker identity is represented via one-hot vectors. In the textual domain, syntactic features are automatically derived with spaCy, encompassing both discrete categorical markers (part-of-speech tags and dependency labels) and continuous structural attributes (token tree depth, total tree depth, word position, and utterance length), complemented by 100-dimensional static fastText embeddings serving as a non-contextual lexical proxy. Combining these diverse sources forms the Full feature ensemble \(Y_{\text{Full}}\), establishing the baseline for maximum explainable variance.
2. Unpooled Frame-level and Subword-level Alignment: Preserving Native Temporal Granularity Without Imposed Pooling Standard probing methodologies in speech often average or pool multi-frame hidden states over word boundaries, which inadvertently imposes text-centric boundaries onto speech models and destroys fine-grained local acoustic dynamics. The encoding probe operates strictly at the native tokenization level of each modality: speech models (wav2vec 2.0 variants) are probed at the level of individual 20ms/25ms acoustic frames, while text models (BERT) are probed at the subword token level. Word-level annotations (syntactic structures and fastText embeddings) are mapped directly onto frames using WebMAUS forced-alignment timestamps. Silent frames are filtered, and frames are sampled randomly without pooling, ensuring that the target vectors \(X\) represent pure, unaltered intermediate hidden activations.
3. Multivariate Ridge Encoding Regression Reconstruction: Linear Mapping with Regularized Variance Accounting To evaluate the degree to which interpretable features explain internal representations, the probe fits an \(L_2\)-regularized multivariate ridge regression model \(g: Y \to X\) using scikit-learn. For activation matrix \(X \in \mathbb{R}^{N \times D}\) and input feature matrix \(Y \in \mathbb{R}^{N \times K}\), the probe learns weights that minimize squared reconstruction error, with the regularization parameter \(\alpha\) optimized over a logarithmic grid \(\{10^n \mid n \in \mathbb{Z}, -3 \le n \le 5\}\) via cross-validation. Reconstruction accuracy is quantified via Unexplained Variance (UV), defined as the ratio of the residual sum of squares to the total sum of squares: $$ \text{UV}(g) = \frac{\text{SS}{\text{res}}}{\text{SS}_i|}}} = 1 - R^2 = \frac{\sum_{i=1}^N |x_i - \hat{x2^2}{\sum $$ Under this formulation, }^N |x_i - \bar{x}|_2^2\(\text{UV} = 0\) indicates flawless linear reconstruction of the model representation space, while \(\text{UV} = 1\) indicates that the feature ensemble fails to explain any variance beyond the sample mean vector.
4. Systematic Feature Ablation and Marginal Contribution Quantification: Decoupling Collinear Covariates via Variance Gaps To isolate the true contribution of a specific property \(Y_i\) over and above other correlated variables, the framework compares the full probe \(g_{\text{Full}}: Y_{\text{Full}} \to X\) against an ablated probe \(g_{\setminus Y_i}: Y_{\text{Full}} \setminus Y_i \to X\). The relative contribution is measured directly by the resulting upward gap in unexplained variance: $$ \Delta \text{UV}(Y_i) = \text{UV}(g_{\setminus Y_i}) - \text{UV}(g_{\text{Full}}) $$ If ablating a feature (such as syntax) triggers a substantial increase in UV even when correlated lexical features remain in the input, that feature necessarily contributes unique, non-redundant variance to the target representation. Conversely, if UV remains unchanged after feature ablation, its apparent decodability in traditional probes is revealed as an artifact of collinearity. The validity of these gaps is further corroborated by permutation controls (shuffling feature blocks along the sample axis to preserve dimensionality) and PCA dimensionality-matching controls.
Key Experimental Results¶
Main Results¶
To demonstrate how classic decoding probes fail to provide a unified scale for comparing feature importance, the authors first evaluated held-out decoding performance across all 12 hidden layers of wav2vec2-base on LibriSpeech train-clean-100, contrasting speaker identification against phonetic classification:
| Layer & Feature Probed | Accuracy Acc. (%) | Balanced Acc. Bal. acc. (%) | Macro-F1 (%) | Methodological Limitation |
|---|---|---|---|---|
| Layer 0 (Speaker ID) | 46.7 | 43.7 | 42.4 | Far above random majority baseline (0.4%) |
| Layer 6 (Speaker ID) | 49.3 | 45.7 | 45.7 | Pronounced mid-layer decodability dip |
| Layer 12 (Speaker ID) | 95.3 | 94.5 | 94.9 | Near-perfect decoding, yet relative scale is unknown |
| Layer 0 (Phone label) | 52.5 | 39.9 | 39.5 | Early acoustic layers encode phone primitives |
| Layer 6 (Phone label) | 75.6 | 69.8 | 71.7 | Substantial phonetic structure decoded |
| Layer 9 (Phone label) | 76.4 | 70.3 | 72.2 | Peak balanced phonetic accuracy |
| Layer 12 (Phone label) | 58.7 | 46.3 | 47.2 | Late-layer phonetic decodability drop |
When applying the Encoding Probe to the exact same representations, the relative contribution profile diverges fundamentally from decodability: in wav2vec2-base and ASR-tuned models, ablating acoustics (\(\text{Full} \setminus \text{Acoustics}\)) or phonetics produces substantially larger upward UV gaps across all layers than ablating speaker identity (\(\text{Full} \setminus \text{Speaker}\)). In striking contrast, for the speaker-identification-tuned (SID) model, \(\text{UV}(g_{\text{Full}})\) drops near 0 in layers 9โ12, and ablating speaker identity causes UV to surge close to 1.0, while ablating acoustics or phonetics produces virtually zero gap, demonstrating that top layers in SID models become almost entirely dedicated to speaker identity.
Ablation Study¶
To resolve whether syntactic decodability is merely a byproduct of lexical correlation in language models, the authors conducted linear probing between features alongside systematic encoding ablations on BERT-base and wav2vec 2.0 variants. Table 5 establishes the empirical baseline for direct linear correlations between static fastText word embeddings and syntactic attributes:
| Target Property / Control Setup | Evaluation Metric | Measured Score | Theoretical & Empirical Implication |
|---|---|---|---|
| fastText โ Part-of-Speech (POS) | Accuracy | 0.628 | Strong intrinsic lexical-syntactic correlation |
| fastText โ Dependency Label | Accuracy | 0.417 | Substantial dependency syntax decodable from static words |
| fastText โ Tree Depth / Position | \(R^2\) | 0.012 ~ 0.030 | Hierarchical tree depth is largely unrecoverable from words |
| Speaker ID from Acoustics | Accuracy | 0.061 | 20ms local acoustic descriptors contain minimal speaker cues |
| Speaker ID from Phonetics (PPG) | Accuracy | 0.008 | Phonetic probabilities are virtually orthogonal to speaker ID |
| Speaker ID from Acoustics + PPG | Accuracy | 0.071 | Combined low-level features fail to linearly yield speaker ID |
| PCA Dimensionality Control (Lexicon \(k=25\)) | Retained Relative Gap | ~57% | Retains majority of explanatory power at acoustic dimension |
| PCA Dimensionality Control (Lexicon \(k=10\)) | Retained Relative Gap | ~29% | Absolute UV scales with dimension, confirming gap metric validity |
Key Findings¶
- Divergence of Decodability and Representational Dominance: High decoding accuracy does not imply that a feature dominates the representation space. In the top layers of the SID-tuned model, phonetic labels remain decodable at non-trivial accuracy, yet their ablation in the encoding probe produces negligible change in UV. The decoding probe detects peripheral extractable signals, whereas the encoding probe measures true representational variance.
- Syntax Independently Shapes Model Representations: In both BERT-base and wav2vec2 models, ablating syntactic features consistently increases UV above the Full baseline even when 100-dimensional lexical embeddings are included. This refutes the skeptical hypothesis that syntactic decodability is an artifact of lexical memorization, confirming genuine syntactic encoding.
- Tuning Objectives Restructure Internal Manifolds: Pre-training preserves balanced acoustic and phonetic variance; ASR fine-tuning sharpens phonetic/lexical contributions and suppresses speaker variance; SID fine-tuning collapses the top layers almost entirely onto a low-dimensional speaker subspace, discarding phonetic detail.
Highlights & Insights¶
- Inverting the Analytical Direction: By framing probing as regression-based representation reconstruction (\(Y \to X\)), the method solves the long-standing incommensurability of cross-task decoding metrics by mapping all attributes onto a universal \(1 - R^2\) scale.
- Principled Solution to Feature Collinearity: Leveraging variance decomposition and subset ablation allows the encoding probe to disentangle tightly coupled linguistic properties (e.g., lexical semantics vs. syntactic structure) without relying on ad-hoc control tasks.
- Multi-Modal and Multi-Objective Generality: The paradigm seamlessly spans speech audio and text transformers, demonstrating how objective functions (SSL, ASR, SID) drive representation specialization across layers.
Limitations & Future Work¶
- Linearity Constraint: Implementing the encoding probe with linear ridge regression prioritizes interpretability and avoids probe overfitting, but inherently cannot capture non-linear feature interactions in transformer activations.
- Observational Nature: Like standard probing, the encoding probe establishes correlational associations rather than causal mechanisms governing the model's downstream computation.
- Sensitivity to Feature Inventory and Dimensionality: The total variance explained is upper-bounded by the chosen feature sets, and absolute variance contributions correlate partly with feature dimensionality, necessitating careful controls when comparing across disparate modalities.
Related Work & Insights¶
- vs. Classic Decoding Probes (e.g., Belinkov 2022, Alain & Bengio 2017): Decoding probes assess whether an attribute is extractable from \(X\), leaving feature importance incomparable across tasks; encoding probes assess how much representational variance is explained by \(Y\), establishing a common scale.
- vs. Conditional Probing (Hewitt et al., 2021): Conditional probing controls for a single confounding variable within a decoding setup; encoding probes naturally accommodate simultaneous multivariate controls across arbitrary numbers of continuous and discrete features.
- vs. Structural Probing & RSA (Hewitt & Manning 2019, Kriegeskorte et al., 2008): Structural probes evaluate tree distance correlations and RSA compares pairwise similarity geometries; the encoding probe directly measures the quantitative percentage of variance attributable to specific linguistic structures.
Rating¶
- Novelty: โญโญโญโญโญ Elegant reversal of the probing paradigm grounded in regression variance decomposition.
- Experimental Thoroughness: โญโญโญโญโญ Thorough coverage across modalities, tuning objectives, permutation, and dimensionality controls.
- Writing Quality: โญโญโญโญโญ Rigorous statistical reasoning, crystal-clear motivation, and lucid comparative analysis.
- Value: โญโญโญโญโญ Establishes a foundational, standardized methodology for representation interpretability beyond simple decodability.