Skip to content

HEAL: Anchored Hyperspherical Equilibrium for Multilingual Safety Alignment

Conference: EMNLP 2026
Paper: ACL Anthology
Area: LLM Reasoning / LLM Safety
Keywords: Multilingual Safety Alignment, Hyperspherical Geometry, DPO Auxiliary Loss, Cross-Lingual Transfer, Alignment and Uniformity

TL;DR

Addressing the vulnerability where English-only safety preference data fails to transfer reliably into low-resource languages, this paper proposes HEAL—a lightweight, plug-and-play auxiliary objective that balances a stop-gradient English anchor shrink force with a pairwise non-English dilation force to establish a controlled shell-like equilibrium geometry on the unit hypersphere, substantially improving multilingual refusal and out-of-distribution jailbreak robustness without requiring any translated responses.

Background & Motivation

The global deployment of large language models across diverse linguistic communities has vastly outpaced the development of multilingual safety alignment. State-of-the-art alignment protocols such as Direct Preference Optimization (DPO) and RLHF depend critically on high-quality pairwise preference supervision consisting of chosen versus rejected responses. However, because curating reliable preference datasets is exceptionally labor-intensive and culturally nuanced, these training corpora are produced almost exclusively in English, leaving models deployed in dozens of typologically distant, lower-resource languages critically vulnerable to safety bypasses. Under standard monolingual DPO, the optimization updates only the English forward trajectory while non-English representations remain unconstrained; English-acquired safety transfers merely opportunistically, leaving lower-resource prompts largely unprotected.

To circumvent the prohibitive cost of curating human-annotated preference responses in every target language, recent research has explored representation-level regularization to project English safety constraints across hidden states. The prominent Multilingual Consistency (MLC) baseline penalizes the non-leading singular values of the multilingual representation matrix to encourage a rank-1 collapse toward a shared direction. Nevertheless, this formulation suffers from a critical structural deficiency: the target alignment vector is data-determined as the joint dominant singular vector across all prompt realizations. Consequently, noisy representations inherent to low-resource languages directly perturb and contaminate the shared direction, leading to volatile alignment targets across varying prompts. Furthermore, coercing all language states into a single collinear subspace inevitably strips away the nuanced angular spread necessary for encoding distinct language-specific syntax, register, and lexical semantics.

Drawing insight from contrastive representation learning, representations constrained to a unit hypersphere remain expressive only when balancing alignment (clustering positive samples along shared semantic pathways) with uniformity (preserving marginal dispersion to retain representational capacity). For any safety-relevant prompt, multilingual translations constitute natural positive instances that must lie close enough to inherit English refusal behavior while maintaining sufficient separation to preserve language-specific expressiveness. Core idea: propose HEAL (Hyperspherical Equilibrium for Aligned Languages), an auxiliary loss that couples a stop-gradient English anchor shrink force with a pairwise non-English hinge dilation force, shaping multilingual prompt representations into an English-centered shell-like equilibrium region that enforces strict safety transfer bounds while preventing collapse.

Method

Overall Architecture

HEAL operates as a modular, lightweight auxiliary regularization objective that integrates seamlessly into any DPO-style alignment pipeline without altering its underlying English supervised preference pairs. During training, a safety-critical query is provided simultaneously in \(L\) languages (\(L=10\), comprising English alongside nine typologically diverse non-English languages). For prompt \(i\) and language \(l\), the hidden state vector from the final prompt token at the last decoder layer is mapped via a parameterized projection head and \(L_2\)-normalized onto the \(D-1\) dimensional unit hypersphere \(\mathbb{S}^{D-1}\). While standard DPO optimizes preference log-likelihoods over English chosen/rejected continuations, HEAL acts upon the normalized hyperspherical representations: a shrink term pulls each non-English representation toward a stop-gradient English anchor, while a dilation term repels all non-English representations from one another. At equilibrium, non-English embeddings reside within an English-centered spherical shell, balancing cross-lingual safety alignment with language-specific uniformity. The auxiliary projection head is discarded after training, introducing zero inference-time parameters or latency.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Multilingual Safety Prompts<br/>10 Language Realizations"] --> B["Hyperspherical Projection<br/>Last-token Hidden State Normalized to S^(D-1)"]
    B --> C["Stop-gradient English Anchor Shrink Force<br/>pull: Bounded Non-English to English Alignment"]
    B --> D["Pairwise Non-English Dilation Force<br/>push: Preserved Multilingual Angular Spread"]
    C & D --> E["Hyperspherical Shell Equilibrium<br/>English-Centered Spherical Cap Geometry"]
    E --> F["Joint End-to-End Optimization<br/>Primary DPO Loss + Auxiliary HEAL Loss"]
    F --> G["Aligned Policy Deployment<br/>Projection Head Discarded · Zero Inference Cost"]

Key Designs

1. Stop-gradient English Anchor Shrink Force: Directional Attraction without Noise Contamination To prevent low-resource representation noise from corrupting the cross-lingual safety direction, the shrink objective designates the normalized English representation \(z_{i,\text{en}}\) as a fixed anchor by wrapping it with a stop-gradient operator \(\text{sg}(\cdot)\). This ensures that auxiliary optimization gradients backpropagate exclusively into non-English forward paths, drawing them toward the high-resource anchor without dragging the English target toward noisy non-English representations. The shrink loss is formulated as a margin-based hinge over the non-English index set \(\mathcal{N} = \{1, \dots, L\} \setminus \{\text{en}\}\): $\(\mathcal{L}_{\text{shrink}}^{(i)} = \frac{1}{|\mathcal{N}|} \sum_{l \in \mathcal{N}} \left[ \| z_{i,l} - \text{sg}(z_{i,\text{en}}) \|_2^2 - r^2 \right]_+\)$ The margin \(r^2\) deactivates the attractive force once a non-English representation moves sufficiently close to the anchor, preventing unnecessary collapse. Assuming the local refusal function \(g_\theta(z)\) is \(K\)-Lipschitz continuous on the unit sphere, this geometric margin guarantees a rigorous theoretical bound on cross-lingual safety discrepancy: \(|g_\theta(z_{i,l}) - g_\theta(z_{i,\text{en}})| \le K r\).

2. Pairwise Non-English Dilation Force: Margin-Controlled Repulsion Preserving Uniformity While the shrink force enforces alignment, attraction alone risks over-compressing all non-English representations into a degenerate cluster, obliterating language-specific semantic distinctions. The dilation term counters this by applying a pairwise repulsive hinge between all pairs of non-English representations within the same prompt: $\(\mathcal{L}_{\text{dilate}}^{(i)} = \frac{2}{|\mathcal{N}|(|\mathcal{N}| - 1)} \sum_{\substack{l, l' \in \mathcal{N} \\ l < l'}} \left[ m^2 - \| z_{i,l} - z_{i,l'} \|_2^2 \right]_+\)$ Crucially, English is excluded from this summation so that the anchor acts purely as an attractive reference rather than being repelled by surrounding languages. By penalizing squared Euclidean distances smaller than the margin \(m^2\), this repulsive force restores local uniformity among same-prompt representations, preserving the capacity required for each language to express its distinct vocabulary, grammar, and idiomatic register.

3. Hyperspherical Shell Equilibrium: Feasible Geometry and Single-Stage Training The interplay between the two opposing forces converges to an ideal equilibrium state when both hinge terms are inactive: \(\| z_{i,l} - z_{i,\text{en}} \|_2^2 \le r^2\) and \(\| z_{i,l} - z_{i,l'} \|_2^2 \ge m^2\) for all \(l \neq l'\). Geometrically, this configuration defines an English-centered spherical shell. In an orthogonal subspace of dimension \(D=512\), embedding \(n = |\mathcal{N}| = 9\) non-English vectors as a regular simplex yields an exact theoretical upper bound on the dilation margin for non-empty feasibility: $\(m^2 \le \frac{2n}{n-1} \left( r^2 - \frac{r^4}{4} \right)\)$ Under the default setting \(n=9\) and \(r^2=0.2\), the theoretical maximum for \(m^2\) is approximately \(0.43\), ensuring that the operational margin \(m^2=0.3\) resides well inside the geometrically feasible domain. The full training objective combines standard preference optimization with HEAL auxiliary terms: $\(\mathcal{L}_{\text{HEAL}} = \mathcal{L}_{\text{DPO}} + \frac{\lambda_{\text{aux}}}{B} \sum_{i=1}^B \left( \mathcal{L}_{\text{shrink}}^{(i)} + \alpha_d \mathcal{L}_{\text{dilate}}^{(i)} \right)\)$ With intra-prompt computation, distance evaluation costs only \(\mathcal{O}(BL^2D)\), adding negligible training overhead while leaving inference latency entirely unaffected.

Loss & Training

  • Training Loss: The primary loss is standard Sigmoid DPO with temperature \(\beta=0.1\) evaluated on English pairs \((x, y_c, y_r)\). The auxiliary loss weights are fixed at \(\lambda_{\text{aux}} = 0.8\), \(\alpha_d = 0.5\), with margins \(r^2 = 0.2\) and \(m^2 = 0.3\).
  • Optimization Strategy: Training is conducted end-to-end for 3 epochs with a per-device batch size of 4 using DeepSpeed ZeRO-3. Peak learning rate is \(6 \times 10^{-7}\) scheduled by cosine decay with a 0.1 warmup ratio in bfloat16 mixed precision.
  • Representation Tapping: Normalized representations (\(D=512\)) are tapped from the final token of the last decoder layer \(\ell\) using a two-layer MLP projection head, optimized jointly with the backbone.

Key Experimental Results

Main Results

Evaluation is performed across two open-source backbones (Qwen-2.5-7B-Instruct and Gemma-2-9B-it) on PKU-SafeRLHF across 10 training languages (EN, ZH, RU, JA, AR, BN, SW, UR, PS, KU), reporting average safety rate (Avg %), cross-lingual variance (Var, where lower denotes balanced equity), and pairwise agreement (PAG, where higher denotes response consistency).

Backbone & Model Coverage Safety Avg (%) ↑ Variance Var ↓ Agreement PAG ↑ Low-Resource Tail (SW / PS / KU)
Qwen-2.5-7B-Instruct
Raw (Pretrained) 10 langs 59.55 0.13140 0.5037 SW: 6.11 / PS: 21.11 / KU: 12.22
SDRRL (Self-Distill) 10 langs 70.28 0.00450 0.8412 SW: 64.44 / PS: 69.44 / KU: 61.11
MPO (Reward Margin) 10 langs 65.50 0.05530 0.6979 SW: 4.44 / PS: 59.44 / KU: 42.78
DPO (Monolingual) 10 langs 66.44 0.12440 0.5437 SW: 7.22 / PS: 30.00 / KU: 17.22
MLC (Rank-1 Collapse) 10 langs 97.39 0.00022 0.9737 SW: 96.67 / PS: 95.00 / KU: 98.89
HEAL (Ours) 10 langs 98.83 0.00010 0.9882 SW: 98.33 / PS: 98.89 / KU: 100.0
Gemma-2-9B-it
Raw (Pretrained) 10 langs 81.78 0.06970 0.7390 SW: 94.44 / PS: 41.11 / KU: 18.89
SDRRL (Self-Distill) 10 langs 71.94 0.01223 0.7288 SW: 72.77 / PS: 76.67 / KU: 72.78
MPO (Reward Margin) 10 langs 78.95 0.01790 0.8641 SW: 76.67 / PS: 70.56 / KU: 42.22
DPO (Monolingual) 10 langs 81.52 0.06520 0.7614 SW: 90.91 / PS: 48.66 / KU: 16.58
MLC (Rank-1 Collapse) 10 langs 95.06 0.00011 0.9194 SW: 96.67 / PS: 93.89 / KU: 95.56
HEAL (Ours) 10 langs 98.06 0.00007 0.9663 SW: 97.78 / PS: 97.78 / KU: 98.33

General task capability and robustness against multilingual jailbreaks on MultiJail (both 5 in-distribution and 5 out-of-distribution languages) are presented below (lower ASR is better, higher general capability is better):

Backbone & Paradigm MultiJail ID ASR (%) ↓ MultiJail OOD ASR (%) ↓ MMLU (EN 5-shot) ↑ MMMLU-lite (6 langs) ↑
Qwen-2.5-7B
Raw 17.27 8.06 73.32 60.33
DPO 10.54 3.62 73.29 64.50
MLC 1.90 1.33 73.10 60.37
HEAL (Ours) 0.89 0.89 73.09 60.50
Gemma-2-9B
Raw 4.38 4.13 72.59 63.83
DPO 2.73 2.79 72.76 64.12
MLC 2.66 1.33 72.76 64.67
HEAL (Ours) 2.09 2.03 72.77 64.42

Ablation Study

On Qwen-2.5-7B-Instruct, the shrink margin is fixed at \(r^2=0.2\) while the squared dilation margin \(m^2\) is varied across a parameter sweep, tracking Swahili (SW) safety and jailbreak susceptibility:

Configuration Variant Dilation Margin \(m^2\) PKU-SW Safety (%) ↑ MultiJail-SW ASR (%) ↓ Mechanism & Geometric Behavior
Shrink-only (No Repulsion) \(m^2 = 0.00\) 93.89 2.86 Inactive dilation term; representations over-compress around anchor, degrading SW expressiveness
HEAL (Mild Repulsion) \(m^2 = 0.15\) 96.11 1.90 Pairwise repulsion takes effect, establishing non-degenerate angular dispersion
HEAL (Default Equilibrium) \(m^2 = 0.30\) 98.33 0.63 Optimal shell geometry, perfectly balancing alignment transfer and language separation
HEAL (Excessive Repulsion) \(m^2 = 0.45\) 96.67 1.27 Exceeds theoretical simplex bound (\(\approx 0.43\)), where excessive repulsion disrupts anchor alignment

Key Findings

  • Dual Forces Outperform Pure Attraction: The ablation at \(m^2=0.0\) demonstrates that eliminating the dilation force leads to a steep drop in Swahili safety (falling from 98.33% to 93.89%) and elevates jailbreak vulnerability (ASR increases from 0.63% to 2.86%), directly verifying that attraction-only objectives induce representational collapse.
  • Maximum Gains Concentrated on Lower-Resource Tails: Base models exhibit catastrophic vulnerabilities on low-resource languages such as Swahili, Pashto, and Kurdish (e.g., 6.11%, 21.11%, and 12.22% on raw Qwen). Vanilla DPO offers negligible improvement, whereas HEAL closes this gap entirely, reaching 98.33%, 98.89%, and 100.0% safety while maintaining minimal cross-lingual variance (\(1.0 \times 10^{-4}\)).
  • Zero Degradation on General Linguistic Capabilities: Across both backbones, HEAL preserves English reasoning on MMLU (73.09 vs 73.10) and multilingual knowledge on MMMLU-lite (60.50 vs 60.37 on Qwen) while strengthening out-of-distribution jailbreak defense across unseen languages (Korean, Italian, Javanese, Thai, Vietnamese).

Highlights & Insights

  • Geometric Formulation of the Alignment-Uniformity Dilemma: Instead of enforcing an unconstrained rank-1 singular value collapse, HEAL translates classical hyperspherical contrastive dynamics into an explicit dual-force equilibrium with mathematically tractable feasibility boundaries.
  • Stop-Gradient Anchor as an Asymmetric Regularizer: The simple application of \(\text{sg}(\cdot)\) on the English embedding breaks optimization symmetry, ensuring that high-resource safety signals anchor low-resource representations without suffering reciprocal degradation from noisy low-resource gradients.
  • Frictionless Training with Zero Inference Overhead: The auxiliary projection head and pairwise distance computations operate exclusively during the backward pass within batch prompts, delivering robust safety geometry without requiring test-time translation or inference latency.

Limitations & Future Work

  • Exploration Limited to 7B-9B Parameter Scale: Empirical verification is conducted primarily on dense 7B and 9B instruction-tuned architectures; scaling behaviors on 70B+ models or Sparse Mixture-of-Experts (MoE) architectures remain uncharacterized.
  • Static Margin Heuristics Across Language Families: A single global margin pair \((r^2, m^2)\) is enforced across all languages regardless of linguistic topology or morphological distance from English; learning adaptive language-family-specific margins represents a natural extension.
  • Scope Confined to Text Modalities: The current framework aligns multilingual text representations; extending anchored hyperspherical equilibrium to multimodal (vision-language-audio) safety alignment provides a compelling future research trajectory.
  • vs MLC (Bu et al., 2026): MLC suppresses trailing singular values to force representations into a shared rank-1 subspace, allowing low-resource noise to steer the alignment direction; HEAL establishes an explicit, fixed English anchor and prevents collapse via pairwise repulsive dilation.
  • vs MPO (Zhao et al., 2025): MPO maps observed reward margins from English into pseudo-preference supervision for non-English prompts at the algorithmic level; HEAL directly shapes hidden-state geometry, eliminating dependence on reward model calibration across languages.
  • vs SDRRL (Zhang et al., 2024): SDRRL relies on self-distillation using translated response outputs, incurring substantial generation overhead and translation quality bottlenecks; HEAL performs representation alignment on prompt tokens alone without translating any response text.

Rating

  • Novelty: ⭐⭐⭐⭐⭐ [Translates hyperspherical alignment-uniformity dynamics into an anchored dual-force shell geometry for multilingual alignment]
  • Experimental Thoroughness: ⭐⭐⭐⭐⭐ [Comprehensive cross-backbone benchmarks spanning standard safety, jailbreak attacks, out-of-distribution transfer, and ablations]
  • Writing Quality: ⭐⭐⭐⭐⭐ [Clear mathematical grounding, elegant geometric formulation, and tight alignment between theory and empirical behavior]
  • Value: ⭐⭐⭐⭐⭐ [Provides a practical, label-free solution to multilingual safety disparities with zero inference-time computational cost]