Turning the Spell Around: Lightweight Alignment Amplification via Rank-One Safety Injection¶
Conference: EMNLP 2026
Paper: ACL Anthology
Area: Safety
Keywords: LLM Safety, Alignment Amplification, Rank-One Weight Editing, Refusal Direction, Jailbreak Defense
TL;DR¶
This paper introduces RANK-ONE SAFETY INJECTION (ROSI), a fine-tuning-free method that extracts a one-dimensional safety refusal vector via difference-in-means on contrasting instructions and permanently injects it as a rank-one perturbation into residual stream write matrices, significantly cutting jailbreak vulnerability and re-aligning uncensored models while preserving general utility.
Background & Motivation¶
Large language models possess impressive generality across open-domain tasks, but pre-training on vast uncurated web corpora inevitably endows them with the latent capability to generate harmful and dangerous content. Although post-training alignment techniques such as supervised fine-tuning (SFT) and reinforcement learning from human feedback (RLHF) have become the de facto standard for behavioral steering, aligned models remain fragile against an expanding arsenal of jailbreak vectors (e.g., DAN, adversarial prompt suffixes, multi-turn role-play) and downstream fine-tuning on even completely benign data.
Recent advances in mechanistic interpretability reveal that refusal behavior in decoder-only transformers is surprisingly low-dimensional, mediated predominantly by a single linear direction in the activation space. Adversarial work like representation abliteration demonstrated that projecting out this one-dimensional refusal direction from model weights trivially neutralizes safety guardrails, producing uncensored models. Conversely, defensive attempts to reinforce safety typically lean on compute-intensive adversarial fine-tuning or inference-time activation steering, which respectively suffer from catastrophic utility degradation and runtime inference overhead.
Inspired by these findings, this work inverts the abliteration vulnerability: rather than ablating the refusal subspace to strip safety, can we systematically and permanently inject this direction into model weights? Core idea: extract a one-dimensional safety direction from a small set of contrasting harmful/harmless instructions and apply a lightweight, fine-tuning-free rank-one weight injection directly to all residual stream write projection matrices (\(W_{\text{out}}\)), permanently amplifying refusal robustness and re-aligning uncensored models with negligible side effects.
Method¶
Overall Architecture¶
ROSI operates in two main phases: safety direction extraction in activation space, followed by rank-one weight editing in parameter space. First, using a small set of harmful and harmless instruction pairs, the model computes residual activations across candidate layers to isolate the mean difference vector at the prompt-final token position; for uncensored models that lack native refusal tendencies, a temporary safety system prompt is prepended during the forward pass to elicit distinct geometric separation. Second, the extracted normalized safety direction is combined with the row-mean vector of each residual write projection matrix to form a rank-one update matrix, which is permanently added to the original weights without gradient-based training.
%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
A["Contrasting Instruction Pairs<br/>(Harmful & Harmless Sets)"] --> B["Safety Direction Extraction<br/>Activation difference-in-means & norm"]
B --> C["Weight Mean Projection<br/>Outer product with row-wise weight mean"]
C --> D["Rank-One Weight Injection<br/>Additive update to Attn and MLP write matrices"]
D --> E["ROSI Model<br/>Permanently amplified refusal & preserved utility"]
Key Designs¶
1. Safety Direction Extraction: Isolating Causal Refusal Subspace via Difference-in-Means
To identify the specific linear direction mediating refusal, ROSI leverages the difference-in-means formulation over two small, contrasting datasets: harmful instructions \(\mathcal{D}_{\text{harmful}}\) and harmless instructions \(\mathcal{D}_{\text{harmless}}\). Forward passes are executed to record the residual stream activations \(x_i^{(l)}\) at the final prompt token position across candidate layers \(l\). Computing the empirical means \(\mu^{(l)}\) and \(\nu^{(l)}\) yields the raw direction vector:
The optimal extraction layer \(l^*\) is identified on a small validation set by selecting the layer that maximizes the harm refusal rate while maintaining a KL-divergence of \(\le 0.1\) on benign inputs to prevent over-refusal. The chosen vector is then L2-normalized into unit direction \(\hat{s} \in \mathbb{R}^{d_{\text{model}}}\). Empirically, as few as 50 instruction pairsโand even a single contrasting pairโsuffice to isolate an effective direction.
2. Rank-One Weight Injection: Parameter-Space Hardening Without Fine-Tuning
Rather than ablating \(\hat{s}\) via projection \(W' \leftarrow (I - \hat{s}\hat{s}^\top)W\), ROSI permanently strengthens it by editing every projection matrix that writes into the residual streamโspecifically, the attention output projection \(W_O\) and MLP output projection \(W_{\text{out}} \in \mathbb{R}^{d_{\text{model}} \times d_{\text{input}}}\). The update rule is formulated as a clean rank-one outer product:
Here, \(\alpha\) is a scalar hyperparameter governing injection magnitude, and \(\bar{w} \in \mathbb{R}^{d_{\text{input}}}\) denotes the mean of the row vectors of the original weight matrix \(W_{\text{out}}\). Because \(\bar{w}^\top\) approximates the expected input projection baseline, the update term \(\alpha(\hat{s}\bar{w}^\top)\) imparts a persistent bias along \(\hat{s}\) upon every forward token write, permanently steering intermediate representations toward the refusal boundary without any runtime steering overhead or modified inference kernels.
3. Prompt-Conditioned Re-Alignment: Resurrecting Latent Safety in Uncensored Models
When applied to uncensored models (such as the Dolphin series) whose safety alignment was intentionally ablated during fine-tuning, standard activation subtraction fails because the baseline model yields almost no refusal signals. ROSI resolves this by prepending a explicit safety system prompt during the forward extraction pass, commanding the model to reject harmful tasks. This system prompt cleanly disentangles harmful and harmless activation clusters (demonstrated via increased cluster centroid distance in PCA projections). Once the latent safety vector is captured and injected into the weights, the system prompt is discarded, and the modified model retains strong refusal behavior autonomously.
Loss & Training¶
ROSI is entirely training-free and requires no gradient descent or loss optimization. Calibration is limited to tuning the injection scalar \(\alpha\) and selecting the target layer set. Layer ablation experiments show that injecting into a contiguous window comprising 30%โ40% of layers centered around the extraction layer achieves near-optimal refusal gains, while applying the update across all residual stream write projections delivers consistent cross-architecture robustness.
Key Experimental Results¶
Main Results¶
ROSI was evaluated across eight open-weight aligned models on CATQA (550 harmful prompts across 11 categories judged by Llama Guard 3) for Harm Refusal (HR %), Alpaca (512 benign prompts) for Benign Compliance (BC %), and four major jailbreak attack suites (DAN, HarmBench, WildGuardTest, and WildJailbreak).
| Model | Method | Harm Refusal (HR โ) | Benign Compliance (BC โ) | DAN ASR (โ) | HarmBench ASR (โ) |
|---|---|---|---|---|---|
| Gemma-2B-Instruct | Baseline + ROSI |
98.4% 99.8% (+1.5) |
99.4% 99.0% (-0.4) |
5.3% 1.0% (-4.3) |
42.3% 8.2% (-34.1) |
| Llama-2-7B-Chat | Baseline + ROSI |
99.8% 100.0% (+0.2) |
98.8% 99.8% (+1.0) |
0.0% 0.0% (0.0) |
3.5% 0.1% (-3.4) |
| Llama-3.1-8B-Instruct | Baseline + ROSI |
98.2% 99.1% (+0.9) |
99.6% 99.6% (0.0) |
0.3% 0.0% (-0.3) |
14.8% 1.8% (-13.0) |
| Llama-3.2-1B-Instruct | Baseline + ROSI |
79.5% 92.7% (+13.2) |
99.2% 95.9% (-3.3) |
1.3% 0.0% (-1.3) |
18.7% 7.5% (-11.2) |
| Qwen2.5-3B-Instruct | Baseline + ROSI |
89.8% 99.6% (+9.8) |
99.6% 98.6% (-1.0) |
52.7% 6.7% (-46.0) |
93.7% 61.5% (-32.2) |
| Qwen2.5-7B-Instruct | Baseline + ROSI |
95.8% 100.0% (+4.2) |
100.0% 99.0% (-1.0) |
40.3% 11.7% (-28.6) |
90.7% 36.7% (-54.0) |
| Qwen2.5-14B-Instruct | Baseline + ROSI |
98.9% 100.0% (+1.1) |
100.0% 99.4% (-0.6) |
32.3% 5.0% (-27.3) |
81.2% 43.9% (-37.3) |
| Yi-6B-Chat | Baseline + ROSI |
81.3% 99.5% (+18.2) |
99.6% 97.7% (-1.9) |
52.0% 15.3% (-36.7) |
89.4% 44.6% (-44.8) |
General capabilities were tested on six benchmarks (MMLU, HellaSwag, ARC-Easy, ARC-Challenge, BoolQ, TruthfulQA). Across all models, the average benchmark score difference remained within \(\pm 0.5\%\), confirming that the rank-one safety direction is largely orthogonal to factual knowledge and reasoning capabilities.
Ablation Study¶
Ablations investigated uncensored model re-alignment, vaccination against downstream fine-tuning degradation, cross-model transfer, and data sample efficiency.
| Model / Setting | Configuration Variant | Harm Refusal (HR %) | Key Robustness / Utility Notes |
|---|---|---|---|
| Dolphin3.0-Qwen2.5-3B | Baseline (Uncensored) ROSI (with safety prompt) ROSI (โข without safety prompt) |
50.0% 86.0% (+36.0) 33.6% (-16.4) |
BC 99.6%; DAN ASR drops to 44.0% Without prompt, overlapping activations yield negative transfer |
| Dolphin3.0-Llama3.1-8B | Baseline (Uncensored) ROSI (with safety prompt) ROSI (โข without safety prompt) |
65.8% 100.0% (+34.2) 88.9% (+23.1) |
Full re-alignment to 100.0% safety with 100.0% BC maintained |
| Llama-2-7B-Chat (Dolly 15K FT) | Base (No FT) LoRA FT SafeLoRA LoRA \(\to\) ROSI (post-hoc) ROSI \(\to\) LoRA (vaccination) ROSI \(\to\) SafeLoRA (joint) |
99.8% 82.7% 95.5% 85.5% 98.6% 100.0% |
Standard FT causes catastrophic safety drop (-17.1%) Post-hoc injection cannot fully recover broken manifolds Pre-injection immunizes model against fine-tuning erosion Joint defense achieves complete 100.0% safety preservation |
| Qwen2.5-3B-Instruct (Sample Efficiency) | Base ROSI (1 pair) ROSI (10 pairs) ROSI (50 pairs) ROSI (100 pairs) |
89.8% 99.1% 96.5% 99.5% 98.6% |
A single instruction pair elevates refusal to 99.1% Validates extreme dominance of refusal direction in representation space |
Key Findings¶
- Amplifying Vulnerable Models: ROSI provides dramatic improvements for models with weaker baseline alignment, raising HR by +18.2% on Yi-6B-Chat and +13.2% on Llama-3.2-1B-Instruct, while slashing HarmBench jailbreak success rates by 30 to 50 percentage points.
- Uncensored Model Recovery: Uncensored models retain latent safety circuits; prompt-conditioned extraction allows ROSI to reinstate refusal guardrails (achieving 100.0% HR on Dolphin3.0-Llama3.1-8B) without expensive re-training.
- Safety Vaccination Mechanism: Injecting the ROSI vector prior to downstream parameter-efficient fine-tuning (ROSI \(\to\) LoRA) shields against safety erosion (98.6% vs. 82.7%), and combining it with SafeLoRA attains 100.0% refusal retention.
- Cross-Lingual & Cross-Model Transfer: Safety directions extracted from English prompts generalize effectively to Arabic tests. Furthermore, between models sharing identical hidden dimensions (\(d_{\text{model}}\)), transferring the safety vector from Qwen2.5-32B into Qwen2.5-14B yields even stronger defense than the 14B model's own vector.
Highlights & Insights¶
- Inverting Abliteration into Construction: Translates mechanistic insights from adversarial attacks (direction erasure) into an elegant, low-cost defensive intervention that directly hardens model weights.
- Ultra-Lightweight Rank-One Parameter Update: Avoids matrix inversions, optimization loops, and gradient propagation; injects a simple outer product perturbation in seconds.
- Zero Runtime Overhead: Unlike test-time activation steering or external guardrail models, ROSI requires no customized inference runtime, extra latency, or memory overhead.
Limitations & Future Work¶
- Strictly Rank-One Assumption: The study restricts updates to rank-one perturbations; higher-rank updates or subspace modeling may be required to counter complex multi-turn or compositional attack vectors.
- Complementary Rather than Standalone: While ROSI substantially curtails automated jailbreaks, it does not guarantee absolute invulnerability and should be combined with input filtering and external guardrails in deployment.
- Slight Benign Degradation on Small Models: Models with \(\le 1\text{B}\) parameters exhibit minor benign compliance drops (e.g., -3.3% on Llama-3.2-1B), highlighting the difficulty of cleanly separating safety and utility in constrained capacity regimes.
Related Work & Insights¶
- vs. Arditi et al. (2024) (Abliteration): While Arditi et al. use orthogonal projection \((I - \hat{s}\hat{s}^\top)W\) to remove refusal, ROSI inverts this paradigm by adding \(\alpha \hat{s}\bar{w}^\top\) to reinforce the safety direction.
- vs. SafeLoRA (Hsu et al., 2025): SafeLoRA projects adapter weights during adaptation; ROSI acts as an upstream initialization vaccine, with ROSI \(\to\) SafeLoRA achieving flawless 100.0% retention against fine-tuning degradation.
- vs. Test-Time Activation Steering (AdaSteer / SafeSteer): Activation steering incurs runtime computational overhead and requires customized serving infrastructure; ROSI modifies static weights once with zero inference overhead.
Rating¶
- Novelty: โญโญโญโญโญ Inverts mechanistic representation ablation into a simple, elegant rank-one static weight injection defense.
- Experimental Thoroughness: โญโญโญโญโญ Evaluated across 12 diverse models (8 aligned, 4 uncensored), multiple jailbreak benchmarks, 6 utility tasks, and fine-tuning vaccination scenarios.
- Writing Quality: โญโญโญโญโญ Clean narrative structure, crisp mathematical formulation, and strong conceptual cohesion with mechanistic interpretability.
- Value: โญโญโญโญโญ Offers a highly practical, compute-free tool for open-source model safety hardening and post-hoc un-censoring defense.