Skip to content

What Drives Representation Steering? A Mechanistic Case Study on Steering Refusal

Conference: EMNLP 2026
Paper: ACL Anthology
Code: https://github.com/step-cheng/steer-interp
Area: Interpretability
Keywords: mechanistic interpretability, representation steering, activation patching, refusal steering, OV circuit

TL;DR

This paper introduces an activation patching framework tailored for multi-token autoregressive generation, revealing that refusal steering vectors derived from distinct methodologies rely on functionally interchangeable subnetwork circuits, demonstrating that steering primarily drives behavioral shifts through the attention OV circuit (specifically an input-invariant steering value vector) rather than the QK circuit, and achieving 85%–96% dimension sparsification.

Background & Motivation

Representation steering vectors have emerged as a lightweight, inference-time alignment and behavioral intervention mechanism for large language models (LLMs). By adding directional offsets to intermediate hidden activations, steering has demonstrated competitive empirical performance across mitigating hallucinations, modulating personas and styles, and bypassing or enforcing safety guardrails via refusal steering. However, despite the proliferation of steering methodologies—ranging from heuristic difference-in-means (DIM) to optimization objectives like next-token prediction (NTP) and preference optimization (PO)—there remains a fundamental lack of mechanistic understanding regarding how these vectors interact with internal model components and cascade into distinct generation outcomes.

Conventional mechanistic interpretability techniques, such as activation patching and circuit discovery, have predominantly been confined to standardized single-token classification tasks driven by counterfactual prompt templates. These methods cannot be directly translated to open multi-token autoregressive generation where the input prompts remain identical and divergence is induced solely by an internal latent intervention at an intermediate layer. This methodological barrier prevents practitioners from systematically assessing steering robustness, diagnosing failure modes and behavioral collapse, or deriving principled priors for crafting fine-grained, concept-expressive steering vectors.

This paper tackles this gap by generalizing causal attribution patching to multi-token steered generation and conducting a thorough case study on refusal—a foundational capability in LLM safety and jailbreaking. Core idea: formalize steering interventions across autoregressive generation as representation-level counterfactual attribution patching, reveal that diverse steering methodologies leverage highly overlapping and functionally interchangeable subnetwork circuits, and mathematically prove that steering acts predominantly via the attention OV circuit rather than QK attention pattern matching, enabling up to 96% gradient-based vector sparsification.

Method

Overall Architecture

The proposed framework establishes an end-to-end pipeline to discover and characterize the causal pathways mediating representation steering. First, multi-token Edge Attribution Patching with Integrated Gradients (EAP-IG) is computed between unsteered base representations and steered representations across teacher-forced decoding trajectories to extract end-to-end subgraphs. Next, cross-method circuit interchangeability experiments evaluate whether different vector derivation paradigms utilize shared neural pathways. Subsequently, a rigorous mathematical decomposition of the post-LayerNorm multi-head attention module isolates the input-invariant Steering Value Vector (SVV) inside the OV circuit from the dynamic QK circuit. Finally, dimension-level indirect effect vectors enable causal gradient-based sparsification.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Input Prompt and Autoregressive Token Sequence"] --> B["Multi-Token Causal Attribution Patching<br/>Token-level EAP-IG between base and steered representations"]
    B --> C["Cross-Method Circuit Discovery & Interchangeability<br/>Extract and validate shared circuits across DIM, NTP, and PO"]
    C --> D["Mathematical Decomposition of the Attention OV Circuit<br/>Isolate input-invariant Steering Value Vector SVV"]
    D --> E["Causal Attribution Dimension Sparsification<br/>Filter core subspace via normalized gradient threshold"]
    E --> F["Output: 85-96% Sparsified High-Fidelity Steering Vector"]

Key Designs

1. Multi-Token Causal Attribution Patching: extending circuit discovery across generation trajectories

Classical attribution patching operates on contrasting prompt pairs (\(x, x^*\)). In representation steering, the prompt text is identical; the intervention is the persistent additive vector \(\alpha \cdot s\) injected at layer \(\ell\). The framework models the unsteered activations \(H^\ell_{\text{base}}\) as corrupted and the steered representations \(H^\ell_{\text{steer}} = H^\ell_{\text{base}} + \alpha \cdot S\) as clean, where \(S = ([s] \times N)^\top \in \mathbb{R}^{N \times d}\) tiles the steering vector over sequence length \(N\). Over each decoding position \(p\), indirect effect (IE) of edge \((u, v)\) is approximated using integrated gradients with midpoint quadrature (\(O(1/T^2)\) error): $\(\text{IE}(u, v) \approx \frac{1}{T} \sum_{i=1}^T \left( \frac{\partial m(H^\ell_{\text{base}} + \frac{i - 0.5}{T} \alpha \cdot S)}{\partial v} \right)^\top (u - u^*)\)$ where the importance metric \(m(x')\) evaluates logit differences between greedy steered and unsteered tokens. Masking positions where base and steered outputs agree focuses the causal attribution entirely on tokens where the steering signal flips the model decision.

2. Cross-Method Circuit Discovery and Functional Interchangeability: validating shared neural pathways

To assess whether distinct steering methodologies exploit common mechanisms, circuits are extracted for vectors derived from heuristic difference-in-means (DIM), self-supervised next-token prediction (NTP), and contrastive preference optimization (PO). Across Gemma 2 2B, Llama 3.2 3B, and Qwen 3 8B, minimum-faithful circuits (\(\text{faithfulness} \ge 0.85\)) exhibit pairwise edge overlaps exceeding 90%. Crucially, when evaluating an NTP or PO vector while restricting the active subnetwork to the topology discovered via the DIM vector, steering faithfulness is preserved (\(\ge 85\%\)), whereas random circuits of double the edge count achieve \(<10\%\) faithfulness. This demonstrates that regardless of optimization formulation, refusal steering converges upon a universal, functionally interchangeable circuit backbone.

3. Mathematical Decomposition of the Attention OV Circuit: isolating the input-invariant SVV

An inspection of edge distributions across the minimum-faithful circuits reveals that incoming edges to attention modules heavily favor Value projections, while Query and Key connections are virtually absent. Expanding the post-RMSNorm attention operation analytically yields: $\(\text{Attention}(H^\ell + \alpha \cdot S) = \sum_h \left( A^h D_c \tilde{H}^\ell W_{OV}^h + D_c^h \text{svv}^h(S) \right)\)$ where \(\text{svv}^h(s) = (s \odot \gamma) W_{OV}^h \in \mathbb{R}^d\) represents the Steering Value Vector of attention head \(h\). Because the attention pattern \(A^h\) sums to 1 along rows, \(A^h D_c \tilde{S} = D_c^h \tilde{S}\) holds for identical token rows, decoupling the direct linear contribution of \(s\) from input-dependent contextual weighting. Applying logit lens to individual \(\text{svv}^h\) vectors surfaces explicit refusal concepts ("prohibition", "forbidden", "illegal"), even when the raw steering vector itself lacks interpretable top tokens. Causal ablation confirms that freezing the QK circuit across all layers degrades attack success rate by only 8.83%, whereas ablating the SVV accounts for 73.4% of the massive 64.00% drop caused by freezing the full OV circuit.

4. Causal Attribution Dimension Sparsification: uncovering the minimal steering subspace

By computing node-level indirect effect vectors \(\vec{\text{IE}}_u = \frac{1}{T} \sum_{i=1}^T \frac{\partial m}{\partial u} \odot (u - u^*)\), setting \(u - u^* = s\) at the injection layer yields the element-wise ratio \(r = \vec{\text{IE}} \oslash s\), exactly matching the average gradient with respect to the steering parameter. Zeroing dimensions where \(r_i < \tau\) produces gradient-based sparsified vectors. Hypergeometric tests on the Intersection over Union (IoU) of non-zero coordinates across DIM, NTP, and PO vectors show statistically significant alignment (\(p < 10^{-10}\) across 20%–95% sparsity), confirming that different methodologies converge onto a compact, low-dimensional coordinate subspace.

Key Experimental Results

Main Results

The ablation experiments benchmark unablated steering against freezing/ablating QK attention weights, OV value vectors, the direct SVV term, and direct MLP inputs across Gemma 2 2B (G2), Llama 3.2 3B (L3), and Qwen 3 8B (Q8) using DIM vectors. Attack Success Rate (ASR) is measured on harmless prompts from Alpaca (A, lower is better when inducing refusal) and harmful prompts from JailbreakBench (JBB, higher is better when bypassing refusal).

Ablation Config Model Alpaca ASR (↓) JailbreakBench ASR (↑) Avg % Change in ASR
None (Unablated) Gemma 2 2B 0.00 0.80 —
Llama 3.2 3B 0.03 0.85 —
Qwen 3 8B 0.02 0.83 —
QK Circuit Frozen Gemma 2 2B 0.03 (+3%) 0.74 (-6%) 8.83%
Llama 3.2 3B 0.14 (+11%) 0.70 (-15%)
Qwen 3 8B 0.07 (+5%) 0.70 (-13%)
OV Circuit Frozen Gemma 2 2B 0.57 (+57%) 0.06 (-74%) 64.00%
Llama 3.2 3B 0.99 (+96%) 0.25 (-60%)
Qwen 3 8B 0.49 (+47%) 0.33 (-50%)
SVV Component Ablated Gemma 2 2B 0.35 (+35%) 0.20 (-60%) 47.00%
Llama 3.2 3B 0.89 (+86%) 0.47 (-38%)
Qwen 3 8B 0.19 (+17%) 0.37 (-46%)
Direct MLP Ablated Gemma 2 2B 0.29 (+29%) 0.57 (-23%) 45.17%
Llama 3.2 3B 0.53 (+50%) 0.09 (-76%)
Qwen 3 8B 0.62 (+60%) 0.50 (-33%)

Ablation Study

The table below illustrates the impact of sparsifying the DIM vector on Gemma 2 2B across varying sparsity percentages \(S (\%)\) comparing gradient-based attribution (Grad.), unnormalized IE attribution (IE), random dropout (Drop.), and bottom-\(k\) magnitude pruning (Bot-k) on Alpaca, JailbreakBench, and StrongReject (SR):

Sparsity S (%) Method Alpaca (A) ASR (↓) JailbreakBench (JBB) ASR (↑) StrongReject (SR) ASR (↑) Note
0.0% Dense Baseline 0.000 0.800 0.850 Full dense vector
53.2% Grad. 0.005 0.780 0.843 Near-zero degradation
Drop. 0.030 0.520 0.601 Random dropping degrades noticeably
83.7% Grad. 0.005 0.770 0.831 Strong fidelity retained
IE 0.000 0.770 0.827 Comparable to gradient attribution
Bot-k 0.025 0.610 0.744 Magnitude pruning begins to fail
Drop. 0.875 0.120 0.077 Random baseline collapses
94.9% Grad. 0.065 0.600 0.645 High effectiveness with only ~5% coordinates
Drop. 0.875 0.140 0.125 Complete behavioral failure

Key Findings

  • High circuit localization: Across all three models, retaining only ~8%–11% of model edges (900/8,790 in Gemma 2 2B; 16,100/195,865 in Qwen 3 8B) recovers \(\ge 85\%\) faithfulness, while the complement subnetwork yields near-zero faithfulness.
  • Attention QK circuit insensitivity: Freezing dynamic attention score maps network-wide results in a modest 8.83% drop in steering ASR, whereas cutting OV pathways or isolating the SVV induces 47.0%–64.0% performance collapse.
  • Resolving superposition via SVV: Dense steering vectors that project into uninterpretable noise under raw logit lens (e.g., Llama 3.2 DIM or Gemma 2 NTP) decompose through head-specific SVVs into explicit refusal tokens ("prohibition", "unethical", "cannot"), untangling semantic superposition.

Highlights & Insights

  • Multi-token representation-level causal patching: Generalizing EAP-IG from discrete prompt differences to latent residual shifts establishes an extensible blueprint for analyzing open-ended generation interventions.
  • Input-invariant SVV lens: Factoring out the linear additive bias in attention provides an analytical tool that exposes latent semantic representations without empirical probing or iterative dictionary learning.
  • Universal sparse alignment subspace: Causal gradient attribution demonstrates that alignment steering vectors can be pruned up to 96% with minimal degradation, showing that behavioral control resides in a compact coordinate manifold.

Limitations & Future Work

  • Limitations acknowledged by authors: The study provides in-depth inspection of attention heads but does not exhaustively dissect the fine-grained role of individual MLP neurons, which account for a sizable fraction of circuit edges. Furthermore, steering is evaluated solely at the optimal middle injection layer rather than across multi-layer schedules.
  • Concept scope: Analysis is centered strictly on refusal in the context of safety and jailbreaking; generalizability to reasoning, stylistic nuance, or arithmetic circuits remains unverified.
  • Future directions: Integrating the multi-token patching framework with Sparse Autoencoders (SAEs) to unpack MLP feature contributions and designing native sparse steering interventions to prevent over-steering degradation.
  • vs. Arditi et al. (2024): Arditi et al. identified that refusal is mediated by a single residual direction but treated downstream propagation as a black box; this work uncovers the specific attention heads and subnetwork circuits driving the behavioral shift.
  • vs. Sinii et al. (2025): Sinii et al. analyzed reasoning vectors restricted to the final two transformer layers; this paper extends end-to-end circuit discovery to middle injection layers where steering is most effective.
  • vs. Traditional circuit discovery (Wang et al., 2023; Mueller et al., 2025): Replaces discrete counterfactual prompt templates with continuous latent interpolation over multi-token autoregressive decoding.

Rating

  • Novelty: ⭐⭐⭐⭐⭐ [Extends attribution patching to multi-token steered generation with rigorous mathematical decomposition]
  • Experimental Thoroughness: ⭐⭐⭐⭐⭐ [Evaluated across Gemma, Llama, and Qwen, spanning three vector training paradigms and four benchmarks]
  • Writing Quality: ⭐⭐⭐⭐⭐ [Exemplary mathematical clarity, well-structured causal narrative, and intuitive visualizations]
  • Value: ⭐⭐⭐⭐⭐ [Crucial insights for both mechanistic interpretability and practical, low-degradation representation steering]