Embracing Anisotropy: Turning Massive Activations into Interpretable Control Knobs for Large Language Models¶

Conference: ACL2026
arXiv: 2603.00029
Code: https://github.com/nyancat0222/dimension-analyzer
Area: Interpretability / Activation steering
Keywords: anisotropy, massive activations, domain-critical dimensions, activation steering, jailbreaking

TL;DR¶

This paper reinterprets the "massive activations," often regarded as outliers in LLMs, as interpretable domain-critical dimensions. By identifying these dimensions through a training-free activation magnitude criterion and performing activation steering exclusively on them, the authors demonstrate superior performance in domain adaptation and jailbreaking scenarios compared to full-dimensional steering.

Background & Motivation¶

Background: Hidden representations in Transformer-based LLMs are typically highly anisotropic, meaning a few dimensions possess significantly higher activation magnitudes than others. Most prior work treated this phenomenon as an imbalance in the representation space or a quantization/stability issue, aiming to suppress outlier dimensions or make representations more isotropic.

Limitations of Prior Work: Treating extreme dimensions solely as noise or artifacts ignores their potential functional roles. Conversely, interpretability methods like probe classifiers and Sparse Autoencoders (SAEs) can link internal representations to semantic concepts but usually require additional training and introduce new parameters and interpretability biases.

Key Challenge: Massive activations appear to be outliers but might actually be sparse functional units formed for domain specialization. The challenge lies in identifying these dimensions without additional training and verifying that they are both interpretable and capable of controlling model behavior.

Goal: The authors aim to prove two points: first, a small number of hidden dimensions are highly critical for performance in specific domains; second, these dimensions can serve as sparse control knobs for finer-grained activation steering.

Key Insight: Starting from 57 subjects in MMLU, the authors treat each subject as a domain, sampling identification and evaluation sets respectively. They first use masking to prove that a single dimension can significantly impact domain performance, then employ simple activation statistics to approximately identify these domain-critical dimensions.

Core Idea: Instead of eliminating anisotropy, one should embrace it: use activation magnitude to find domain-critical dimensions, then manipulate only these dimensions to achieve domain adaptation or behavioral guidance.

Method¶

The central hypothesis of this paper is that "extreme activations are not pure noise, but signatures of functional specialization." Based on this, the authors designed a two-part workflow: identifying domain-critical dimensions followed by Critical Dimension Steering (CDS) targeting these dimensions. Note that the local cache only covers up to Section 2.2; steering details and full limitations are not in the cache. The following is based only on confirmable content.

Overall Architecture¶

Given a pre-trained LLM, a target domain, and several domain samples, the method first collects activation statistics for each dimension across all hidden states. For MMLU, 100 test prompts are used per subject, with 50 as an identification set to find critical dimensions and 50 as an evaluation set to verify their impact. The identification phase does not train probes or SAEs; instead, it selects top-\(k\) dimensions based on activation magnitude and domain-discriminative activation frequency. The control phase uses these dimensions as sparse steering targets, modifying only the identified critical dimensions rather than applying uniform intervention to the entire hidden vector.

Key Designs¶

Verifying Dimensional Sparse Criticality via Masking:
- Function: To prove that not all hidden dimensions are equivalent and that a few dimensions possess a decisive influence on performance for certain subjects.
- Mechanism: For Gemma-2-2B-IT and Qwen-3-8B, the authors zero out activations of specific dimensions layer by layer and measure the accuracy drop on the evaluation set. The input embedding layer is excluded to focus on internal processing dimensions.
- Design Motivation: If masking a single dimension barely affects performance, the "critical dimension" hypothesis would be invalid. However, experiments show that zeroing a single dimension can cause Qwen-3-8B's average accuracy to drop significantly, indicating that dimensional importance is highly sparse.
Identifying Domain-Critical Dimensions via Activation Magnitude:
- Function: To find functionally critical and domain-discriminative dimensions using training-free statistical criteria.
- Mechanism: A dimension is considered "active" for a query if its activation deviates from the mean by more than \(3\sigma\). The activation frequency of each dimension within a subject is calculated. If the difference in activation frequency between two domains exceeds 30%, the dimension exhibits a domain-discriminative pattern. Top-\(k\) high-magnitude dimensions are then selected as domain-critical dimensions.
- Design Motivation: Activation magnitude is a signal already formed by the model itself, requiring no new training. If high-magnitude dimensions overlap with ground-truth critical dimensions identified by masking, it suggests functional importance can be inferred from statistical features.
Critical Dimension Steering (CDS):
- Function: To use identified sparse dimensions as precise behavioral control knobs.
- Mechanism: While traditional activation steering applies directions to the entire hidden vector, CDS intervenes only on the top-\(k\) domain-critical dimensions while leaving others untouched. Although full implementation details are missing from the cache, the abstract and introduction state that CDS is applied to domain adaptation and jailbreaking.
- Design Motivation: If domain behavior is dominated by a few high-impact dimensions, full-dimensional steering would disturb numerous irrelevant dimensions. Sparse steering is more likely to achieve strong control with fewer side effects and higher interpretability.

Loss & Training¶

The proposed method is a training-free interpretability and inference-time steering approach, introducing no new training losses. The identification phase utilizes hidden activation statistics from MMLU subject prompts. The verification phase evaluates performance via masking, domain adaptation accuracy, and jailbreak attack success rates. Specific steering coefficients or layer selection strategies for CDS are not confirmable from the cache.

Key Experimental Results¶

Main Results¶

The main tables in the cache primarily show the impact of single-dimension masking on accuracy and summarize CDS effects mentioned in the abstract/introduction.

Experiment	Model	Baseline/Control	Key Result	Description
Single-dim masking	Gemma-2-2B-IT	Original Acc 56.53%	After Rank-1 masking: 41.97%	Mean drop of 14.56 pts
Single-dim masking	Gemma-2-2B-IT	Original Acc 56.53%	After Rank-10 masking: 52.39%	Effect weakens with lower rank
Single-dim masking	Qwen-3-8B	Original Acc 73.30%	After Rank-1 masking: 21.97%	Mean drop of 51.33 pts
Single-dim masking	Qwen-3-8B	Original Acc 73.30%	After Rank-100 masking: 71.97%	Minimal effect for most dimensions
Domain Adaptation	MMLU 57 subjects	Whole-dimension steering	CDS better on 34/57 subjects	Aggregate from introduction
Jailbreaking	AdvBench	Whole-dimension: 84% ASR	CDS reaches 92% ASR	Aggregate from abstract

Ablation Study¶

The local cache does not contain full ablation tables. Confirmable evidence relies on functional sparsity and domain discriminatity.

Object	Settings (Confirmable)	Observation	Conclusion
Functional Sparsity	Layer-wise zero-masking of a dimension	Masking Rank-1 dim in Qwen-3-8B drops accuracy from 73.30% to 21.97%	A few dimensions dominate domain performance
Domain Discriminatity	Active if value \(> 3\sigma\) from mean	Dimensions exist with \(>30\%\) frequency difference between Math and Biology	Extreme activations carry domain information
Interpretability Case	Gemma-2-2B-IT token-level patterns	Dim 1046 activates for Math terms; 2106 for Bio; 334 for keywords	Single dimensions act as semantic detectors

Key Findings¶

The impact of a single hidden dimension can be massive. In Qwen-3-8B, masking the most critical dimension drops average accuracy from 73.30% to 21.97%.
High-magnitude dimensions are not merely unstable outliers; in MMLU subjects, they correspond to differences in domains like Mathematics and Biology.
Aggregate results for CDS show that sparse steering outperforms whole-dimension steering in 34/57 MMLU subjects and achieves a 92% ASR in AdvBench jailbreaking compared to 84%.
Due to cache limitations, subject-level accuracy, layer effects, or top-\(k\) sensitivity cannot be confirmed.

Highlights & Insights¶

The most compelling aspect is the paradigm shift: while anisotropy was previously treated as a representational defect to be calibrated, this paper views it as a natural consequence of internal specialization.
Using magnitude to identify critical dimensions is simple yet effectively captures the observable signals of massive activations. This makes the method more lightweight than probes/SAEs and suitable for rapid diagnosis.
CDS connects interpretability with control: if a dimension explains a domain concept, manipulating only those dimensions acts as a "precision knob" compared to full-dimensional steering.
There is a security implication: sparse dimensions that increase jailbreak ASR are also weak points in the model's safety boundary, where interpretability tools and attack tools converge.

Limitations & Future Work¶

The full limitations section is missing from the cache. The following are inferred from the available experimental design.
Only aggregate descriptions for MMLU and AdvBench are visible; the lack of subject-level tables or layer ablations makes it difficult to judge the stability boundaries of CDS.
"Domains" are operationalized by MMLU subjects, which works for exam topics but may not cover mixed domains, long contexts, or open-ended generation in real-world applications.
While the training-free magnitude criterion is convenient, it might mistake "high-frequency formatting features" for semantic functional units. Future work should align these with causal interventions, SAE features, or probe results.
The use of jailbreak ASR to verify control capability implies a potential dual-use risk; public tools should include safety analysis.

vs. isotropy calibration / outlier suppression: Those works tend to reduce the influence of extreme dimensions; this paper argues they are meaningful functional units and should be interpreted before deciding to suppress them.
vs. probe classifier: Probes learn conceptual directions but require supervision and training. This paper directly reads activation statistics, offering lower costs and shorter explanation chains.
vs. sparse autoencoder: SAEs decompose polysemantic hidden features into sparser interpretable features. This paper searches for critical dimensions within the original hidden space, making it lighter but more limited in expressive power.
vs. whole-dimension activation steering: Traditional steering affects the entire vector, potentially introducing irrelevant perturbations. CDS limits intervention to verified causal handles by modifying only critical dimensions.

Rating¶

Novelty: ⭐⭐⭐⭐☆ The perspective is distinct, reframing massive activations as "interpretable control units" with a simple method.
Experimental Thoroughness: ⭐⭐⭐☆☆ Key evidence and aggregates are present, but the lack of full steering tables and ablations makes it hard to judge robustness.
Writing Quality: ⭐⭐⭐⭐☆ The introduction clearly explains the background and motivation.
Value: ⭐⭐⭐⭐☆ Insightful for interpretability, activation steering, and security auditing, particularly for lightweight internal representation diagnosis.