Measuring Affinity between Attention-Head Weight Subspaces via the Projection Kernel¶
Conference: EMNLP 2026
Paper: ACL Anthology
Code: https://github.com/ymgw55/proj-kernel-attn-subspace
Area: LLM Reasoning
Keywords: Mechanistic Interpretability, Attention Head Subspaces, Projection Kernel, Composition Score, Identity Heads
TL;DR¶
Addressing the direction-dependent scaling and norm distortion problems of the classical Composition Score, this work introduces the principal-angle-based Projection Kernel (PK) to measure the geometric overlap between attention-head weight subspaces directly, substantially improving head circuit detection and functional clustering across GPT2, BLOOM, and Qwen while identifying critical hub identity heads.
Background & Motivation¶
Mechanistic interpretability seeks to decode the reverse-engineered computational circuits underlying language models. Contemporary circuit analysis predominantly relies on dynamic, input-dependent techniques—such as activation patching and causal interventions applied to the Indirect Object Identification (IOI) benchmark. While these microscopic approaches can isolate localized computational paths on curated prompt templates, their discoveries are fundamentally conditioned on the chosen textual domain and vulnerable to distribution shifts, offering limited visibility into the global, task-agnostic geometric properties encoded within the static parameters.
To characterize model mechanisms without running inference across infinite inputs, weight-based similarity metrics have emerged as an appealing alternative. The foundational paradigm is the Composition Score (CS) formulated by Elhage et al., which normalizes the Frobenius norm of inter-layer matrix products between query-key (QK) and output-value (OV) matrices. However, CS exhibits fundamental mathematical flaws: its singular value decomposition reveals that the metric inherently weights singular directions by the magnitude of their corresponding singular values. Consequently, a handful of dominant singular directions disproportionately dictate the score, obscuring the genuine geometric alignment of the multi-dimensional subspaces. Furthermore, CS lacks invariance under invertible transformations and is severely distorted by layerwise fluctuations in matrix norms—such as the systematic swelling of OV norms and attenuation of QK norms across deeper layers in GPT2-small—which artificially inflates denominators and penalizes deep inter-layer connections.
To resolve these distortions, this paper re-envisions head-to-head interaction capacity through a purely geometric lens. The capacity of an attention head to read representations written by preceding heads is fundamentally bounded by the geometric alignment of the \(d_{\text{head}}\)-dimensional subspaces spanned by their weight matrices, rather than the arbitrary scale of their weights. Core idea: by substituting singular-value-scaled matrix norms with the principal-angle-based Projection Kernel (PK), we measure the unweighted, rotation-invariant geometric overlap between attention-head weight subspaces, establishing a rigorous framework complete with theoretical null distributions and topological hub detection.
Method¶
Overall Architecture¶
The proposed analytical pipeline proceeds through four modular, sequential phases: First, column-orthonormal basis matrices spanning the \(d_{\text{head}}\)-dimensional subspaces are constructed for the query (Q), key (K), value (V), and output (O) weight matrices of all attention heads. Second, the Projection Kernel is computed across all pairwise head combinations over all 16 pairing types by summing the squared cosines of their principal angles (equivalently evaluating the trace of the product of their orthogonal projectors). Third, to ground subspace geometry in linguistic meaning, vocabulary tokens are interpreted by projecting centered and normalized unembedding vectors onto each head's subspace. Fourth, empirical PK distributions are benchmarked against an analytical null distribution derived on the Stiefel manifold to quantify informativeness, while topological inlet and outlet scores isolate information hubs across layers.
%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
A["Multi-Head Attention Weights<br/>W_Q, W_K, W_V, W_O ∈ R^(d × d_head)"] --> B["Principal-Angle Projection Kernel<br/>Orthonormal Bases U, U' Projector Inner Product tr(P P')"]
B --> C["Subspace Vocabulary Projection<br/>Centered & Normalized Projection P f(e_t)"]
B --> D["Inlet and Outlet Hub Identification<br/>Max-PK Transmission Scores for Routing Hubs"]
B --> E["Theoretical Null & Informativeness Framework<br/>KL Divergence against Random Stiefel Subspaces"]
C --> F["Interpretable Circuit Wiring & Head Roles"]
D --> F
E --> F
Key Designs¶
1. Principal-Angle Projection Kernel: Eliminating Direction-Dependent Scale Biases The classical Composition Score normalizes matrix products by their Frobenius norms, which under compact SVD decomposes into an inner product of left and right singular vectors weighted by normalized singular value coefficients. This creates an acute direction-dependent bias toward the highest singular values. To overcome this, the authors employ the Projection Kernel (PK) based on principal angles. Given two \(d_{\text{head}}\)-dimensional subspaces \(\mathcal{S}, \mathcal{S}' \subset \mathbb{R}^d\) spanned by orthonormal basis matrices \(U, U' \in \mathbb{R}^{d \times d_{\text{head}}}\), and their orthogonal projection matrices \(P = UU^\top, P' = U'U'^\top\), the \(i\)-th principal angle \(\theta_i\) corresponds directly to the \(i\)-th singular value \(\sigma_i\) of \(M = U^\top U'\). PK is formulated as the sum of squared cosines of all principal angles:
Algebraically, PK is mathematically equivalent to computing the Composition Score directly on the orthogonal projection matrices: \(PK(\mathcal{S}, \mathcal{S}') = (d_{\text{head}} CS(P', P))^2\). Because projection matrices possess uniform eigenvalues (either 0 or 1), PK eliminates singular-value scale distortion entirely. For toy intersecting subspaces, PK evaluates exactly to the integer dimension of their intersection \(|\mathcal{I} \cap \mathcal{I}'|\), exhibiting natural invariance to arbitrary orthogonal rotations \(\Psi \in \mathcal{O}(\mathbb{R}^d)\) and avoiding sensitivity to weight pre-multiplication.
2. Subspace Vocabulary Projection: Removing High-Norm Artifacts via Centered Normalization Prior attempts to interpret head weights using the Logit Lens or singular vectors often directly project the unembedding matrix \(E_{\text{out}}\), yet this is confounded by the pronounced anisotropy of word embeddings, where rarely generated tokens and corrupted "glitch tokens" have disproportionately large Euclidean norms that dominate projections. For each head weight subspace \(\tilde{\mathcal{S}}_r\) calibrated by the final LayerNorm, the authors design a rigorous preprocessing map \(f(e_t)\) for token embedding \(e_t\). Observing that the mean token embedding \(\bar{e}\) points strongly in the direction opposite to the final LayerNorm bias \(\beta_{\text{final}}\) (\(\cos(\bar{e}, \beta_{\text{final}}) = -0.431\)), centering eliminates this dominant artifact before normalization:
This transformation decouples token norms from projected logits (\(\rho = -0.146\)), suppressing meaningless glitch tokens while preserving a strong correlation (\(\rho = 0.998\)) with the generative bias direction, thus exposing clean, human-interpretable top vocabulary items (e.g., proper nouns for Name Movers, numerical successor sequences for Induction Heads).
3. Inlet and Outlet Hub Identification: Uncovering Intermediate Identity Heads Visualizing top PK edges in GPT2-small revealed specific intermediate heads that act as major topological convergence nodes. To formalize this structural role, the authors introduce the PK Inlet Score and Outlet Score. For an operation \(\omega \in \{Q, K, V\}\), the inlet score measures the degree to which head \(h\) in layer \(\ell_h\) serves as the global maximum PK recipient among all downstream candidates for signals emitted by upstream heads \(h'\):
The outlet score mirrors this by summing instances where an upstream head achieves the maximum PK score toward downstream targets. In GPT2-small, head L4H7 records both the highest OV inlet score (228.05) and highest OK outlet score (263.86). Empirical verification confirms that L4H7 is an Identity Head whose attention pattern is nearly an identity matrix along the diagonal, functioning as a high-fidelity information relay that reads previous activations and transmits them downstream.
4. Theoretical Null Distribution & Informativeness Framework: Statistical Baselines for Subspace Overlap Because different weight pairing types (such as OQ, OK, OV, or QQ) naturally exhibit divergent dynamic ranges, determining whether an observed affinity is statistically meaningful requires an uncorrupted null hypothesis. Under the Stiefel manifold \(\mathcal{V}_m(\mathbb{R}^d)\), the authors leverage random matrix theory to derive the exact analytical mean and variance of the Projection Kernel between two uniformly distributed random orthogonal subspaces:
Invoking the central limit theorem yields a tight Gaussian approximation \(\mathcal{N}(\mathbb{E}[Z], \text{Var}(Z))\). By calculating the Kullback-Leibler (KL) divergence between empirical PK distributions and this theoretical baseline, the informativeness of all 16 pairing types is systematically compared. Weight pairings where output acts as the source (OQ, OK, OV) and homogeneous pairings (QQ, KK) exhibit massive KL divergences (up to 2.876 for QQ), whereas non-functional pathways like VO collapse toward the random baseline (KL = 1.149), providing formal statistical verification of structural asymmetry in Transformers.
Benchmark & Evaluation Protocol¶
The methodology is evaluated across three distinct language models: GPT2-small (124M, 12 layers, 12 heads MHA, LayerNorm), BLOOM-560M (560M, 24 layers, 16 heads MHA, ALiBi), and Qwen2-0.5B-Instruct (494M, 24 layers, 14 query / 2 KV heads GQA, RMSNorm, SwiGLU). Using the documented IOI circuit head annotations as ground-truth labels (comprising 36 positive heads in GPT2-small across Duplicate Token, Previous Token, Induction, Name Mover, Negative Name Mover, Backup Name Mover, and S-Inhibition classes), two evaluation protocols are established: 1. Annotated Head Detection: Measures whether ranking head pairs by inter-layer similarity (9,504 candidate pairs in GPT2-small) effectively discriminates known circuit-participating heads from irrelevant heads, measured by PR-AUC. 2. Homogeneous Classwise Classification: Evaluates whether heads within the same functional class (e.g., Induction Heads) share tighter intra-class subspace overlaps across QQ, KK, VV, and OO pairings compared to cross-class pairs, evaluated by class-averaged PR-AUC and ROC-AUC under extreme class imbalance.
Key Experimental Results¶
Main Results: Circuit Head Detection and Functional Classification¶
In the annotated head detection benchmark (Table 1 & Table 4), the Projection Kernel dramatically outperforms the Composition Score and geometric baselines, establishing substantial gains across OQ and OK pairings. In homogeneous classwise classification (Table 2 & Table 5), PK maintains superior precision in highly imbalanced regimes.
| Model | Evaluation Task | Metric | PK (Ours) | CS | Simple-CS | Linear CKA | Procrustes Sim. |
|---|---|---|---|---|---|---|---|
| GPT2-small | OQ Head Detection | PR-AUC | 0.446 | 0.227 | 0.245 | 0.337 | 0.328 |
| GPT2-small | OK Head Detection | PR-AUC | 0.451 | 0.352 | 0.359 | 0.323 | 0.269 |
| GPT2-small | OV Head Detection | PR-AUC | 0.289 | 0.215 | 0.305 | 0.409 | 0.428 |
| GPT2-small | Mean (OQ/OK/OV) | PR-AUC | 0.395 | 0.265 | 0.303 | 0.356 | 0.342 |
| GPT2-small | Homogeneous Pairing | PR-AUC | 0.047 | 0.013 | 0.016 | 0.021 | 0.038 |
| GPT2-small | Homogeneous Pairing | ROC-AUC | 0.809 | 0.785 | 0.792 | 0.715 | 0.787 |
| BLOOM-560M | Mean (OQ/OK/OV) | PR-AUC | 0.268 | 0.141 | 0.224 | 0.193 | 0.147 |
| BLOOM-560M | Homogeneous Pairing | PR-AUC | 0.027 | 0.003 | 0.015 | 0.001 | 0.001 |
| BLOOM-560M | Homogeneous Pairing | ROC-AUC | 0.866 | 0.844 | 0.869 | 0.533 | 0.568 |
| Qwen2-0.5B | Mean (OQ/OK/OV) | PR-AUC | 0.251 | 0.190 | 0.222 | 0.092 | 0.091 |
| Qwen2-0.5B | Homogeneous Pairing | PR-AUC | 0.032 | 0.005 | 0.004 | 0.024 | 0.024 |
| Qwen2-0.5B | Homogeneous Pairing | ROC-AUC | 0.781 | 0.804 | 0.794 | 0.509 | 0.557 |
Ablation Study: Unembedding Preprocessing Dynamics¶
Evaluating the correlation between token unembedding norms and projected logit magnitudes across all 144 heads in GPT2-small (Table 7) demonstrates how preprocessing choices impact vocabulary interpretability. Unnormalized projections are heavily biased by raw token norms, whereas centered normalization cleanly decouples norm artifacts.
| Preprocessing Function \(f(e_t)\) | Baseline Norm | Query Head \(\rho\) | Key Head \(\rho\) | Value Head \(\rho\) | Output Head \(\rho\) | All Heads \(\rho\) | Interpretability Impact |
|---|---|---|---|---|---|---|---|
| Raw \(e_t\) | \(\|e_t\|\) | \(0.750 \pm 0.049\) | \(0.757 \pm 0.045\) | \(0.745 \pm 0.063\) | \(0.670 \pm 0.091\) | \(0.731 \pm 0.074\) | Severe norm bias; dominated by rare/glitch tokens |
| Centered \(e_t - \bar{e}\) | \(\|e_t - \bar{e}\|\) | \(0.605 \pm 0.084\) | \(0.625 \pm 0.078\) | \(0.670 \pm 0.108\) | \(0.547 \pm 0.147\) | \(0.612 \pm 0.116\) | Global mean removed, but norm disparity remains high |
| Normalized \(e_t / \|e_t\|\) | \(\|e_t\|\) | \(-0.392 \pm 0.092\) | \(-0.363 \pm 0.086\) | \(-0.382 \pm 0.127\) | \(-0.402 \pm 0.145\) | \(-0.385 \pm 0.116\) | Eradicates correlation with bias \(\beta_{\text{final}}\) (\(\rho=0.045\)) |
| Centered + Norm (Ours) | \(\|e_t - \bar{e}\|\) | \(-0.198 \pm 0.138\) | \(-0.168 \pm 0.139\) | \(-0.054 \pm 0.226\) | \(-0.163 \pm 0.185\) | \(-0.146 \pm 0.184\) | Preserves bias alignment (\(\rho=0.998\)); pure semantic signal |
Key Findings¶
- High-Score Regime Divergence: While PK and CS exhibit strong macro-level rank correlation across all 9,504 head pairs (\(\rho \approx 0.84 - 0.88\)), their Spearman correlation drops precipitously when restricted to the top 3% candidate pairs (\(-0.251\) for OQ, \(-0.110\) for OK, and \(0.186\) for OV). CS prioritizes head pairs with isolated dominant singular values that fail to form cohesive circuits.
- Circuit Layer Flow and Clean Wiring: In GPT2-small wiring diagrams, the top 20 PK edges accurately reconstruct the known IOI circuit, showing concentrated information routing from Previous Token heads (L4H11, L5H6) into Induction heads, and S-Inhibition head L8H10 routing directly into Name Mover L9H6 and Backup Name Movers L10H1 and L10H6. Conversely, CS diagrams connect early-layer unannotated heads to deep layers, producing excessive noise.
- OV Channel Complexity: Across all models, OV pairing detection yields lower PR-AUC under PK compared to Linear CKA and Procrustes Similarity. This reflects the multi-functional semantic transformation performed by value-to-output projections, where information routing interacts heavily with residual stream preservation and intermediate identity hubs.
Highlights & Insights¶
- Scale-Free Geometric Interpretability: By formalizing head interactions through the Grassmannian projection kernel rather than Frobenius norm ratios, this paper demonstrates that attention head communication is governed by multi-dimensional subspace directions, resolving years of heuristic normalizations in the mechanistic interpretability community.
- Hub Discovery Without Training Dynamic Probing: The discovery of L4H7 as an Identity Head highlights the power of topological analysis on static parameters. Without running inference datasets or prompt sweeps, extreme values in inlet/outlet scores autonomously identified an essential routing mechanism in GPT2-small.
- Rigorous Random Matrix Null Baseline: The derivation of the analytical Gaussian distribution for random orthogonal subspaces provides an objective standard of significance, allowing researchers to rigorously determine which inter-module pathways carry structured computational flow versus random weight drift.
Limitations & Future Work¶
- Omission of MLP Blocks and Layer Normalization: Like the original Composition Score, the pairwise formulation models head interactions in isolation, omitting non-linear feedforward networks (MLPs) and LayerNorm transformations between intermediate layers.
- Architectural Challenges with GQA and MLA: The formulation assumes independent \(d_{\text{head}}\)-dimensional weight projections for each head. In models with grouped-query attention (e.g., Qwen2), sharing key and value projections creates identical inlet scores and ties, while low-rank compressed architectures like DeepSeek MLA cannot be parsed without structural adaptations.
- Correlation Versus Causal Execution: While PK indicates the structural bandwidth available for information exchange between subspaces, it cannot verify whether specific tokens or prompt conditions actually activate that pathway during autoregressive generation.
Related Work & Insights¶
- vs Elhage et al. (2021) [Composition Score]: Elhage et al. pioneered weight composition using Frobenius norms of QK and OV products. This paper proves mathematically that CS suffers from direction-dependent scaling and demonstrates that PK serves as the proper scale-invariant metric on projection operators.
- vs Merullo et al. (2024) [Talking Heads] & Ahmad et al. (2025): Previous works performed SVD on weight matrices to isolate individual 1D singular vectors. PK treats the entire \(d_{\text{head}}\)-dimensional subspace holistically, avoiding arbitrary singular vector truncation.
- vs Wang et al. (2023) [IOI Circuit Analysis]: While Wang et al. relied on extensive empirical ablation and activation patching across test datasets to isolate the IOI circuit, this work proves that the same circuit backbone can be uncovered directly from static weight geometry.
Rating¶
- Novelty: ⭐⭐⭐⭐⭐ [Introduces principal-angle projection kernels to Transformer weight analysis, uncovering the mathematical flaws of CS and defining inlet/outlet hub metrics]
- Experimental Thoroughness: ⭐⭐⭐⭐⭐ [Extensively verified across GPT2, BLOOM, and Qwen, validated against IOI ground truth, and supported by full random matrix derivations]
- Writing Quality: ⭐⭐⭐⭐⭐ [Exemplary mathematical clarity, rigorous proofs in appendices, and compelling visual wiring diagrams]
- Value: ⭐⭐⭐⭐⭐ [Provides a robust, scale-invariant foundation for static model auditing and automated mechanistic circuit discovery]