Skip to content

CODA: Mitigating Over-Refusal and Jailbreaks in Large Language Models via Codebook-Orthogonal Direction Alignment

Conference: EMNLP 2026
Paper: ACL Anthology
Area: LLM Reasoning
Keywords: LLM Safety, Activation Steering, Vector Quantization Codebook, Soft Orthogonalization, Over-Refusal Mitigation

TL;DR

To resolve the critical tension between jailbreak vulnerability and benign over-refusal in inference-time activation steering, CODA projects hidden states onto a vector-quantized discrete codebook to extract stable semantic anchors, softly orthogonalizes the safety direction against benign technical intent, and dynamically modulates residual-stream intervention via risk-aware gating and exponential temporal decay without modifying model weights.

Background & Motivation

Large Language Models (LLMs) have achieved unprecedented reasoning and generation performance across diverse downstream domains, yet their open-ended capabilities make them acutely susceptible to malicious prompt manipulations and adversarial jailbreaks. Mainstream safety alignment paradigms primarily rely on post-training procedures—such as Reinforcement Learning from Human Feedback (RLHF), Direct Preference Optimization (DPO), and constitutional supervision—to hardcode safety constraints into model parameters. However, post-training is computationally prohibitive, and models aligned via standard post-training remain inherently brittle against evolving adversarial jailbreaks, including deceptive role-play personas and adversarial suffix perturbations. To bypass the heavy computational overhead of repeated retraining, recent efforts have actively explored inference-time control paradigms, most notably activation steering, which extracts contrastive refusal or safety directions from internal representations and injects them during decoding to steer model generation back toward compliant trajectories.

Despite these promising advantages, conventional continuous activation steering methods are trapped in a severe dilemma between jailbreak mitigation and benign over-refusal. This failure stems directly from the geometric entanglement of harmful and benign technical representations within dense, continuous activation spaces. In typical language representations, queries about cyber-security exploits, malicious hacking, low-level systems programming, and benign administrative debugging inevitably share overlapping latent coordinate axes. When practitioners inject a coarse-grained refusal or safety steering vector to block adversarial jailbreaks, this continuous intervention unconditionally spills over into benign technical semantics, triggering rampant over-refusal where safe, legitimate technical queries are rejected outright. Conversely, dampening the steering magnitude to preserve technical utility severely compromises jailbreak robustness. This fundamental coupling highlights the urgent need for a more selective, disentangled representation control interface.

Drawing inspiration from sparse dictionary learning, which successfully untangles superposed interpretable features from dense representations, this paper views jailbreak vulnerability and over-refusal as coupled manifestations of directional entanglement in continuous space. Rather than manipulating entangled native layers directly, the authors propose mapping hidden representations into a discrete, structured codebook geometry. Core idea: project prompt hidden states onto spherical vector-quantized codebook anchors to estimate stable semantic directions, softly orthogonalize the safety direction against benign technical intent to eliminate geometric interference, and apply the decoded steering vector via risk-aware gating and exponential temporal decay during early decoding steps.

Method

Overall Architecture

CODA (Codebook-Orthogonal Direction Alignment) operates entirely at inference time while keeping the target LLM parameters completely frozen. The overall framework is decomposed into two decoupled phases: an offline direction calibration phase and an online risk-gated inference phase. In the offline phase, target-layer hidden representations of small calibration datasets—spanning harmful adversarial prompts, safe general instructions, and benign technical queries—are extracted and mapped via a lightweight MLP encoder into a spherical vector-quantized (VQ) codebook. In this discrete latent space, class centroids and corresponding directional vectors are computed. CODA then performs soft orthogonalization to remove the projection component aligned with benign technical intent before decoding the residual steering vector back to the model's native hidden dimension. During online inference, the prompt representation is evaluated against the decoded safety direction via an empirical risk gate; the intervention is subsequently injected through a forward hook at the designated layer during the early decoding window with exponential temporal decay.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Calibration Prompts & User Input<br/>D_harm / D_safe / D_tech"] --> B["Vector-Quantized Codebook Bottleneck<br/>Unit-sphere nearest-neighbor quantization"]
    B --> C["Codebook-Space Direction Estimation<br/>Compute v_harm, v_safe, and v_tech"]
    C --> D["Benign-Technical Soft Orthogonalization<br/>Remove parallel technical interference component"]
    D --> E["Dynamic Risk Gate Assessment<br/>Calculate prompt-level risk score g(x)"]
    E --> F["Temporally Decayed Residual Injection<br/>Inject over early T_inj window with exp decay"]
    F --> G["Compliant and Utility-Preserved Text Generation"]

Key Designs

1. Vector-Quantized Codebook Bottleneck: Eliminating Representation Perturbations and Stabilizing Direction Estimation Directly estimating contrastive vectors in continuous activation space exhibits high variance due to superficial variations such as prompt length, phrasing style, vocabulary frequency, and activation norms. CODA introduces a discrete vector-quantized bottleneck parameterized by a two-layer MLP encoder \(\text{Enc}: \mathbb{R}^d \to \mathbb{R}^{d'}\) that maps the last-token prompt hidden state \(h(x)\) at target layer \(l^*\) into a latent vector \(z_e\). Both the latent representation and the learnable codebook entries \(C = [e_1, \dots, e_K] \in \mathbb{R}^{d' \times K}\) (\(K=64\)) are \(L_2\)-normalized onto the unit sphere as \(\tilde{z}_e = z_e / \|z_e\|_2\) and \(\tilde{e}_k = e_k / \|e_k\|_2\). Nearest-neighbor spherical quantization assigns the latent vector to its closest prototype: $\(k^* = \arg\min_{k \in \{1,\dots,K\}} \|\tilde{z}_e - \tilde{e}_k\|_2^2, \quad z_q(x) = \tilde{e}_{k^*}\)$ Because minimizing Euclidean distance on the unit sphere is mathematically equivalent to maximizing cosine similarity (\(\|\tilde{z}_e - \tilde{e}_k\|_2^2 = 2 - 2\langle \tilde{z}_e, \tilde{e}_k \rangle\)), clustering depends strictly on angular semantic alignment rather than activation magnitude. This induces a stable Voronoi partition on the unit sphere, replacing intra-class sample variance with between-cell covariance \(\Sigma_q^c \preceq \Sigma_c\), substantially suppressing estimator variance.

2. Codebook-Space Direction Estimation: Decoupling Malicious, Safe, and Technical Class Centroids Standard steering methods compute a coarse difference vector between mixed harmful and harmless prompts. CODA instead leverages three disjoint calibration sets: adversarial harmful prompts \(\mathcal{D}_{harm}\), general benign instructions \(\mathcal{D}_{safe}\), and benign technical queries \(\mathcal{D}_{tech}\) prone to over-refusal. By mapping calibration hidden states through the frozen encoder and spherical VQ module, CODA computes the empirical centroids in codebook space as \(\mu_c = \mathbb{E}_{x \sim \mathcal{D}_c}[z_q(x)]\) for \(c \in \{harm, safe, tech\}\). The contrastive directions are then isolated relative to the safe baseline: $\(v_{harm} = \mu_{harm} - \mu_{safe}, \quad v_{tech} = \mu_{tech} - \mu_{safe}, \quad v_{safe} = -v_{harm} = \mu_{safe} - \mu_{harm}\)$ This formulation captures explicit directional relationships separating adversarial threats, standard compliance, and over-refusal-sensitive technical tasks in a compact latent geometry.

3. Benign-Technical Soft Orthogonalization: Geometrically Decoupling Interference from Technical Utility Whenever the safety steering direction \(v_{safe}\) has a non-zero inner product with the benign technical direction \(v_{tech}\) (\(\langle v_{safe}, v_{tech} \rangle \neq 0\)), applying safety steering inevitably perturbs benign technical queries into refusal mode. To eliminate this geometric crosstalk, CODA softly orthogonalizes \(v_{safe}\) by subtracting a fraction \(\gamma \in [0, 1]\) of its projection onto \(v_{tech}\): $\(v'_{safe} = v_{safe} - \gamma \frac{\langle v_{safe}, v_{tech}\rangle}{\|v_{tech}\|_2^2} v_{tech}\)$ The resulting direction is normalized to \(\hat{u}_z = v'_{safe} / \|v'_{safe}\|_2\) and decoded back into native residual space using the lightweight two-layer MLP decoder \(\text{Dec}: \mathbb{R}^{d'} \to \mathbb{R}^d\): $\(\hat{u} = \frac{\text{Dec}(\hat{u}_z)}{\|\text{Dec}(\hat{u}_z)\|_2}\)$ When \(\gamma = 0\), the formulation recovers standard mean-difference steering; when \(\gamma = 1\), full orthogonalization ensures that any prompt displaced purely along the technical axis receives an inner-product score of exactly zero (\(\langle \hat{u}_z, v_{tech} \rangle = 0\)), mathematically eliminating technical-axis leakage.

4. Dynamic Risk Gate and Early Exponential-Decay Injection: Calibrating Strength and Generation Windows Applying static steering uniformly across all inputs degrades conversational fluency and downstream knowledge reasoning on safe queries. CODA computes a prompt-level scalar risk score \(r(x) = -\langle h(x), \hat{u} \rangle\), reflecting alignment with harmful intent. Using empirical percentile cutoffs calibrated on a held-out split, \(\tau_{safe} = P_{85}\{r(x) \mid x \in \mathcal{D}_{safe}\}\) and \(\tau_{harm} = P_{10}\{r(x) \mid x \in \mathcal{D}_{harm}\}\), CODA constructs a bounded, risk-adaptive gating coefficient: $\(g(x) = \text{clip}\left(\sqrt{\frac{r(x) - \tau_{safe}}{\tau_{harm} - \tau_{safe}}}, 0, 1\right)\)$ Because initial autoregressive tokens dictate whether generation follows an unsafe compliance or compliant refusal trajectory, CODA confines intervention to an early decoding window \(t \le T_{inj}\) (typically \(T_{inj}=30\)) while applying exponential temporal decay with rate \(\lambda\): $\(\tilde{h}_t^{(l^*)} = h_t^{(l^*)} + \alpha_{max} g(x) e^{-\lambda t} \hat{u}, \quad \text{for } t \le T_{inj}\)$ For all subsequent decoding steps \(t > T_{inj}\), the intervention disengages completely, allowing the model to generate responses with unperturbed fluency and depth.

Loss & Training

The target LLM \(f_\theta\) remains completely frozen throughout all stages. The lightweight VQ module (two-layer MLP encoder and decoder with latent dimension \(d'=1024\), intermediate dimension 2048, LayerNorm, and GELU) is trained on frozen hidden states extracted from candidate layers using the Straight-Through Estimator (STE). The training objective combines mean-squared reconstruction error with commitment loss: $\(\mathcal{L}_{VQ} = \|h - \text{Dec}(z_q^{st})\|_2^2 + \beta_{vq} \|\text{sg}[z_q] - \tilde{z}_e\|_2^2\)$ where \(z_q^{st} = \tilde{z}_e + \text{sg}[z_q - \tilde{z}_e]\), \(\text{sg}[\cdot]\) represents the stop-gradient operator, and the commitment weight is set to \(\beta_{vq} = 0.25\). Codebook vectors are updated via Exponential Moving Average (EMA) with a decay factor of 0.99, accompanied by dead-code revival. Target layer \(l^*\), maximum steering scale \(\alpha_{max}\), and orthogonalization weight \(\gamma\) are selected on a validation set (e.g., \(l^*=16, \alpha_{max}=120, \gamma=0.25\) for Qwen2.5-7B-Instruct; \(l^*=21, \alpha_{max}=0.20^\dagger\) relative to activation norm for LLaMA3.1-8B-Instruct).

Key Experimental Results

Main Results

Evaluation spans four open-weight LLMs: Qwen2.5-7B-Instruct, Qwen3-8B, Qwen3-14B, and LLaMA3.1-8B-Instruct. Benchmarks comprise a 300-prompt jailbreak pool (JailbreakBench, Safety-Prompts, JailBench), a 150-prompt XSTest technical split for over-refusal, a C-Eval validation subset for general knowledge utility, and a 300-prompt ShareGPT split for conversational fluency. Evaluated metrics include Attack Success Rate (ASR↓), benign Compliance Rate (CR↑), XSTest compliance (XSTest↑), C-Eval accuracy (C-Eval↑), ShareGPT chat score (Chat↑), and Harmonic Mean (HMean↑). Values represent averages over three independent runs.

Model Method ASR(%)↓ CR(%)↑ XSTest(%)↑ C-Eval(%)↑ Chat(pts)↑ HMean↑
Qwen2.5-7B Base 95.3 50.0 49.3 76.3 6.1 17.7
Sys. Prompt 51.0 43.3 40.7 75.2 6.1 51.2
CAA 24.3 94.0 95.3 73.5 6.6 79.1
Refusal Direction 30.0 20.7 21.3 70.4 5.5 35.3
AdaSteer 27.3 92.0 93.3 73.1 6.5 77.5
SAS (Static CODA) 16.3 94.7 95.3 73.7 6.4 80.3
CODA (Dynamic) 12.7 93.3 96.7 74.6 6.5 81.5
Qwen3-8B Base 69.0 55.3 60.7 69.0 5.4 50.1
Sys. Prompt 3.7 42.0 37.3 76.3 6.3 55.6
CAA 9.0 48.7 35.3 56.3 6.7 54.0
Refusal Direction 39.7 0.7 0.0 45.6 6.3 0.0
AdaSteer 13.3 63.3 59.3 70.8 5.7 65.9
SAS (Static CODA) 29.0 88.0 89.3 69.0 5.4 71.9
CODA (Dynamic) 22.3 77.3 62.7 69.0 5.4 67.0
Qwen3-14B Base 97.8 65.3 64.0 77.9 5.4 9.7
Sys. Prompt 24.5 38.7 32.0 82.5 6.3 50.8
CAA 3.8 57.3 56.0 71.7 6.1 65.7
Refusal Direction 8.7 46.7 54.7 80.6 5.9 62.4
AdaSteer 5.4 49.3 52.7 75.0 5.7 61.9
SAS (Static CODA) 7.1 57.3 56.7 77.9 5.4 64.7
CODA (Dynamic) 6.5 57.3 56.7 77.9 5.4 64.8
LLaMA3.1-8B Base 65.3 100.0 100.0 46.0 6.0 57.3
Sys. Prompt 28.0 32.7 30.0 43.5 6.3 42.8
CAA 32.7 0.0 0.0 0.0 2.2 0.0
Refusal Direction 3.3 2.0 6.7 0.0 3.3 0.0
AdaSteer 21.0 50.0 48.7 43.7 6.3 54.4
SAS (Static CODA) 60.3 100.0 100.0 46.0 6.0 59.8
CODA (Dynamic) 60.7 99.3 99.3 46.0 6.0 59.5

Ablation Study

Ablations on Qwen2.5-7B-Instruct systematically evaluate the contributions of the VQ codebook bottleneck, soft orthogonalization, and the risk gate. Cross-backbone evaluation further isolates the impact of soft orthogonalization.

Qwen2.5-7B Component Ablation:

Group Variant VQ Soft-Orth Gate ASR(%)↓ CR(%)↑ C-Eval(%)↑
Gate utility test (\(\alpha_{max}=160\)) CODA ✓ ✓ ✓ 25.0 96.0 76.2
w/o Risk Gate ✓ ✓ × 17.7 99.3 48.8
Direction-quality test (\(\alpha_{max}=120\)) CODA ✓ ✓ ✓ 26.0 90.7 75.8
w/o Soft-Orth ✓ × ✓ 50.3 92.7 75.6
w/o VQ (Continuous Latent) × ✓ ✓ 31.0 86.7 75.8

Cross-Backbone Effect of Soft Orthogonalization:

Backbone Method ASR(%)↓ CR(%)↑ XSTest(%)↑ C-Eval(%)↑ Chat(pts)↑ HMean↑
Qwen2.5-7B CODA 12.7 93.3 96.7 74.6 6.5 81.5
w/o Soft-Orth 50.3 92.7 96.7 75.6 6.4 71.3
Qwen3-8B CODA 22.3 77.3 62.7 69.0 5.4 67.0
w/o Soft-Orth 24.3 49.3 49.3 68.7 5.7 58.3
Qwen3-14B CODA 6.5 57.3 56.7 77.9 5.4 64.8
w/o Soft-Orth 9.2 52.7 52.7 80.0 5.9 63.6
LLaMA3.1-8B CODA 60.7 99.3 99.3 46.0 6.0 59.5
w/o Soft-Orth 0.7 26.7 31.3 36.7 5.5 40.0

Key Findings

  • Soft Orthogonalization is Essential for Safety Calibration: Removing soft orthogonalization on Qwen2.5-7B nearly doubles ASR from 26.0% to 50.3%. On LLaMA3.1-8B, omitting orthogonalization causes severe over-refusal, plummeting CR to 26.7% and reducing HMean from 59.5 to 40.0. This confirms that eliminating the parallel technical component directly unlocks effective safety control.
  • Risk Gating Protects Core Model Capabilities Under High Strength: Under aggressive steering (\(\alpha_{max}=160\)), eliminating the prompt-level gate drops C-Eval accuracy from 76.2% down to 48.8%. The dynamic gate scales exposure appropriately, preventing catastrophic interference on benign queries.
  • Discrete Quantization Imparts Representation Robustness: Estimating directions directly in continuous space (w/o VQ) worsens both ASR (31.0% vs. 26.0%) and benign compliance (86.7% vs. 90.7%). Under adversarial text rewrites (politeness framing and length padding), codebook directions rotate substantially less (reducing angular rotation by 40%–80%), proving the stabilizing power of discrete centroid modeling.

Highlights & Insights

  • Spherical Vector Quantization as an Interpretable Control Bottleneck: Employing spherical nearest-neighbor quantization filters out continuous superficial noise (prompt length, lexical frequency) and anchors latent features to discrete geometric centroids, significantly lowering estimation variance.
  • Closed-Form Geometric Disentanglement via Soft Orthogonalization: Rather than hoping multi-task training will separate intent, CODA analytically removes the benign technical component from the safety vector in codebook space, decoupling over-refusal from jailbreak suppression at the geometric root.
  • Temporal Locality in Autoregressive Steerability: Leveraging the insight that the compliance/refusal trajectory is decided within the first few decoding steps, confining intervention to an early window with exponential decay provides effective protection while leaving later long-form generation intact.

Limitations & Future Work

  • Gate Attenuation in Multi-Turn Conversational Escalation: In multi-turn progressive jailbreaks (Crescendo attacks), as conversation length increases, the prompt-level dot-product gate produces lower risk scores (dropping from 0.9 in turn 1 to <0.1 in later turns), leading to under-intervention when malicious goals are eventually introduced.
  • Vulnerability to Complex Context Camouflage: In obfuscated single-turn attacks employing steganography or framing camouflage, shared framing vectors obscure the linear risk gate, reducing gate AUROC from 87.7% to 56.5%.
  • Lack of Cross-Model Transferability: Transplanting safety directions between backbones with identical hidden dimensions (e.g., Qwen3-8B and LLaMA3.1-8B) results in severe performance degradation, confirming that steering vectors remain tied to model-specific latent geometries.
  • vs CAA (Rimsky et al., 2024): CAA computes difference vectors directly in dense continuous activation space, causing severe over-refusal on technical queries under high intervention strength; CODA stabilizes direction estimation via a VQ codebook and prevents technical over-refusal via soft orthogonalization.
  • vs Refusal Direction (Arditi et al., 2024): Relying on a single unconstrained refusal direction indiscriminately shuts down benign queries, yielding near-zero compliance on models like Qwen3-8B (0.7% CR) and LLaMA3.1-8B (2.0% CR); CODA preserves near-perfect compliance through adaptive gating.
  • vs AdaSteer (Zhao et al., 2025b): AdaSteer fits dual refusal and harmfulness directions in continuous space, yet remains susceptible to geometric entanglement; CODA operates over discrete Voronoi cells, achieving a superior safety–utility Pareto frontier across all four evaluated backbones.

Rating

  • Novelty: ⭐⭐⭐⭐⭐ [Pioneering use of discrete VQ codebooks to resolve directional entanglement in activation steering]
  • Experimental Thoroughness: ⭐⭐⭐⭐⭐ [Extensive evaluations across 4 backbones, covering single-turn, multi-turn, ablations, causal probes, and human audits]
  • Writing Quality: ⭐⭐⭐⭐⭐ [Rigorous geometric proofs, intuitive visualizations, and lucid conceptual prose throughout]
  • Value: ⭐⭐⭐⭐⭐ [A plug-and-play, training-free safety alignment framework offering clean separation between safety and utility]