HEAL: Anchored Hyperspherical Equilibrium for Multilingual Safety Alignment¶
Conference: EMNLP 2026
Paper: ACL Anthology
Area: LLM Safety
Keywords: Multilingual Safety Alignment, Hyperspherical Representation Learning, Preference Optimization, Contrastive Equilibrium, Jailbreak Defense
TL;DR¶
HEAL addresses the severe cross-lingual disparity in safety alignment by introducing an auxiliary objective that pulls non-English representations toward a stop-gradient English anchor while enforcing pairwise hinge repulsion between languages, substantially boosting low-resource safety rates without requiring multilingual response supervision.
Background & Motivation¶
The global deployment of large language models (LLMs) has substantially outpaced the development of balanced multilingual safety mechanisms. Prevailing preference optimization frameworks, such as Direct Preference Optimization (DPO), rely almost exclusively on high-quality English chosen-versus-rejected response pairs. In practice, however, real-world deployment covers dozens of typologically diverse languages. Applying standard monolingual DPO refines refusal behaviors on English queries effectively, yet it lacks any built-in mechanism to propagate these safety boundaries to lower-resource representations; moreover, curating comprehensive multilingual response preference datasets is economically prohibitive and practically unscalable for the low-resource languages that need protection the most.
To bypass this data bottleneck, recent research has explored hidden-state representation alignment, attempting to transfer safety policies into non-English prompts via latent geometry rather than additional response supervision. However, representation-level objectives face the classic alignment-uniformity trade-off observed in contrastive learning: representations of the same prompt across different languages should be sufficiently close so that safety supervision transfers across them, but not so close that language-specific grammatical, register, and idiomatic nuances are wiped out. Prior methods occupy opposite extremes of this spectrum. Vanilla DPO leaves non-English hidden states unconstrained, yielding negligible cross-lingual transfer, whereas recent rank-1 consistency approaches (such as MLC) drive all off-leading singular values of the multi-language prompt matrix to zero. This rank-1 collapse suffers from a structural flaw: the shared convergence direction is dynamically computed across all languages jointly, allowing the noisy representations of low-resource languages to contaminate the alignment target and induce harmful cross-prompt instability.
Drawing inspiration from hyperspherical equilibrium geometry, this paper departs from unconstrained collapse and instead introduces a margin-controlled shell manifold around an anchored reference. Core idea: Anchor cross-lingual safety transfer to a stop-gradient English reference (shrink force) while explicitly repelling same-prompt language representations via a pairwise hinge (dilation force), establishing a balanced hyperspherical equilibrium that guarantees safety propagation while strictly preserving language-specific expressiveness.
Method¶
Overall Architecture¶
The HEAL architecture introduces a lightweight, auxiliary representation-level objective that seamlessly augments standard DPO pipelines without altering the underlying preference supervision. For any given safety prompt presented across \(L=10\) languages (English plus 9 non-English languages), the chosen and rejected response pairs remain exclusively in English, driving the standard DPO policy loss. In parallel, prompt tokens pass through the LLM backbone, and the hidden state at the last token of a selected decoder layer is extracted and mapped onto a unit hypersphere \(\mathbb{S}^{D-1}\) via a learned projection head followed by \(L_2\) normalization. On this hypersphere, HEAL applies two opposing geometric forces: an anchored shrink force pulls each non-English representation toward a frozen English anchor, while a dilation force enforces pairwise hinge repulsion across all non-English representations. At inference time, the projection head is discarded entirely, incurring zero additional runtime parameters or latency.
%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
A["Multilingual Safety Prompt Input<br/>(English + 9 Non-English)"] --> B["LLM Backbone Forward Pass<br/>Extract Last-Token Hidden States"]
B --> C["Projection & Normalization<br/>Map to Unit Hypersphere"]
C --> D["English-Anchored Shrink Force<br/>Stop-gradient attraction guides transfer"]
D --> E["Non-English Pairwise Dilation Force<br/>Hinge margin repulsion prevents collapse"]
E --> F["Hyperspherical Shell Equilibrium<br/>Joint optimization with DPO"]
Key Designs¶
1. English-Anchored Shrink Force: Asymmetric pull preventing target drift Previous representation consistency techniques (e.g., MLC) identify the target direction from the dominant singular vector across all language representations jointly, allowing unstable representations in low-resource languages to corrupt the shared alignment direction. To ensure that safety boundaries learned on English transfer stably without reciprocal degradation, HEAL anchors the attractive force to the English representation using a stop-gradient operator. For prompt \(i\), non-English representations in set \(\mathcal{N}\) are pulled toward the frozen English anchor \(\text{sg}(z_{i,\text{en}})\) via a margin-hinged Euclidean distance: $\(\mathcal{L}_{\text{shrink}}^{(i)} = \frac{1}{|\mathcal{N}|} \sum_{l \in \mathcal{N}} \left[ \|z_{i,l} - \text{sg}(z_{i,\text{en}})\|_2^2 - r^2 \right]_+\)$ The stop-gradient operator ensures that auxiliary gradients never backpropagate through the English branch, preventing the anchor from drifting toward noisy low-resource representations. Concurrently, the squared margin \(r^2\) turns off the attraction once a representation enters the target radius, avoiding unnecessary point-collapse onto the anchor.
2. Non-English Pairwise Dilation Force: Margin-bounded spread preserving linguistic capacity Applying shrink alone risks over-compressing all non-English representations into an identical point around \(z_{i,\text{en}}\), wiping out language-specific features required to encode distinct typology and vocabulary, ultimately degrading base language comprehension. To preserve representational diversity, HEAL incorporates a pairwise dilation hinge that penalizes any pair of non-English representations \(l, l' \in \mathcal{N}\) when their Euclidean distance falls below margin \(m\): $\(\mathcal{L}_{\text{dilate}}^{(i)} = \frac{2}{|\mathcal{N}|(|\mathcal{N}| - 1)} \sum_{l, l' \in \mathcal{N}, l < l'} \left[ m^2 - \|z_{i,l} - z_{i,l'}\|_2^2 \right]_+\)$ Crucially, the English anchor is excluded from this pairwise repulsion sum so that it functions strictly as an attractive target rather than a dynamic node subjected to repulsion. This design guarantees an explicit angular separation among non-English representations on the hypersphere, preserving the representational capacity of individual languages.
3. Hyperspherical Shell Equilibrium: Provable feasibility and Lipschitz transfer bounds At equilibrium, when both the shrink and dilation hinge losses are fully inactive, all non-English representations reside inside an English-centered spherical shell region: \(\|z_{i,l} - z_{i,\text{en}}\|_2^2 \le r^2\) and \(\|z_{i,l} - z_{i,l'}\|_2^2 \ge m^2\) for all \(l \neq l'\). Theoretical analysis on \(\mathbb{S}^{D-1}\) shows that by placing non-English vectors along a regular simplex in the subspace orthogonal to \(z_{i,\text{en}}\), this constrained region is strictly non-empty provided \(m^2 \le \frac{2n}{n-1}(r^2 - \frac{r^4}{4})\). With \(n=|\mathcal{N}|=9\) and default \(r^2=0.2\), the theoretical ceiling is \(0.43\), which comfortably satisfies the selected operational margin \(m^2=0.3\). Under this geometry, assuming the local refusal score function \(g_\theta(z)\) is \(K\)-Lipschitz continuous, the safety margin error between non-English and English representations is formally bounded by \(|g_\theta(z_{i,l}) - g_\theta(z_{i,\text{en}})| \le K r\), establishing a controllable theoretical guarantee for safety transfer.
Loss & Training¶
The overall training objective combines standard preference optimization with the two hyperspherical auxiliary forces in an end-to-end, single-stage scheme: $\(\mathcal{L}_{\text{HEAL}} = \mathcal{L}_{\text{DPO}} + \frac{\lambda_{\text{aux}}}{B} \sum_{i=1}^B \left( \mathcal{L}_{\text{shrink}}^{(i)} + \alpha_d \mathcal{L}_{\text{dilate}}^{(i)} \right)\)$ Here, \(\mathcal{L}_{\text{DPO}}\) is computed using the sigmoid DPO loss (\(\beta = 0.1\)) exclusively on English pairs. Default hyperparameters are \(\lambda_{\text{aux}} = 0.8\), \(\alpha_d = 0.5\), \(r^2 = 0.2\), and \(m^2 = 0.3\). The projection head dimension is set to \(D = 512\), tapped at the final decoder layer of the model. Optimization runs for 3 epochs with DeepSpeed ZeRO-3, using a per-device batch size of 4, a peak learning rate of \(6 \times 10^{-7}\) with cosine decay, and a 0.1 warmup ratio under bfloat16 mixed precision. Auxiliary computations are restricted to within-prompt representations, requiring only \(O(BLD) + O(BL^2D)\) distance evaluations with zero cross-prompt pairing overhead.
Key Experimental Results¶
Main Results¶
On the PKU-SafeRLHF safety benchmark (1,800 unsafe prompts across 10 training languages), models are evaluated on per-language safety rates (%), cross-lingual variance (Var, lower is better), and pairwise agreement (PAG, higher is better). HEAL consistently outperforms raw base models, DPO, representation baselines (MLC), and transfer baselines (MPO, SDRRL) across both Qwen-2.5-7B and Gemma-2-9B backbones.
| Model & Method | EN | ZH | RU | JA | AR | BN | SW | UR | PS | KU | Avg (%) โ | Var โ | PAG โ |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Qwen-2.5-7B | |||||||||||||
| Raw | 93.33 | 96.11 | 93.33 | 92.22 | 93.89 | 53.33 | 6.11 | 33.89 | 21.11 | 12.22 | 59.55 | 0.13140 | 0.5037 |
| SDRRL | 57.22 | 73.33 | 77.22 | 75.00 | 77.22 | 72.22 | 64.44 | 75.56 | 69.44 | 61.11 | 70.28 | 0.00450 | 0.8412 |
| MPO | 81.11 | 81.67 | 82.22 | 77.22 | 78.33 | 77.22 | 4.44 | 70.56 | 59.44 | 42.78 | 65.50 | 0.05530 | 0.6979 |
| DPO | 99.44 | 98.33 | 97.22 | 97.78 | 96.11 | 70.56 | 7.22 | 50.56 | 30.00 | 17.22 | 66.44 | 0.12440 | 0.5437 |
| MLC | 96.11 | 98.33 | 97.78 | 98.89 | 98.89 | 98.33 | 96.67 | 95.00 | 95.00 | 98.89 | 97.39 | 0.00022 | 0.9737 |
| HEAL (Ours) | 96.67 | 98.89 | 100.0 | 98.89 | 100.0 | 98.33 | 98.33 | 98.33 | 98.89 | 100.0 | 98.83 | 0.00010 | 0.9882 |
| Gemma-2-9B | |||||||||||||
| Raw | 95.56 | 95.56 | 96.11 | 92.78 | 97.22 | 92.22 | 94.44 | 93.89 | 41.11 | 18.89 | 81.78 | 0.06970 | 0.7390 |
| SDRRL | 51.11 | 78.33 | 82.22 | 50.56 | 73.33 | 81.67 | 72.77 | 80.00 | 76.67 | 72.78 | 71.94 | 0.01223 | 0.7288 |
| MPO | 89.44 | 81.67 | 87.78 | 85.56 | 87.22 | 85.00 | 76.67 | 83.33 | 70.56 | 42.22 | 78.95 | 0.01790 | 0.8641 |
| DPO | 95.00 | 94.59 | 93.55 | 94.62 | 96.77 | 91.98 | 90.91 | 92.51 | 48.66 | 16.58 | 81.52 | 0.06520 | 0.7614 |
| MLC | 93.89 | 96.11 | 95.00 | 96.11 | 93.33 | 95.00 | 96.67 | 95.00 | 93.89 | 95.56 | 95.06 | 0.00011 | 0.9194 |
| HEAL (Ours) | 98.33 | 97.78 | 97.78 | 98.89 | 96.11 | 99.44 | 97.78 | 98.33 | 97.78 | 98.33 | 98.06 | 0.00007 | 0.9663 |
On the MultiJail benchmark (evaluating attack success rate ASR on in-distribution ID languages and 5 out-of-distribution OOD languages) and general capability benchmarks (MMLU and MMMLU-lite):
| Backbone & Method | MultiJail ID ASR (%) โ | MultiJail OOD ASR (%) โ | MMLU (EN) โ | MMMLU-lite (Multi) โ |
|---|---|---|---|---|
| Qwen-2.5-7B | ||||
| Raw | 17.27 | 8.06 | 73.32 | 60.33 |
| DPO | 10.54 | 3.62 | 73.29 | 64.50 |
| MLC | 1.90 | 1.33 | 73.10 | 60.37 |
| HEAL | 0.89 | 0.89 | 73.09 | 60.50 |
| Gemma-2-9B | ||||
| Raw | 4.38 | 4.13 | 72.59 | 63.83 |
| DPO | 2.73 | 2.79 | 72.76 | 64.12 |
| MLC | 2.66 | 1.33 | 72.76 | 64.67 |
| HEAL | 2.09 | 2.03 | 72.77 | 64.42 |
Ablation Study¶
On Qwen-2.5-7B-Instruct, the shrink margin is held constant at \(r^2=0.2\) while the squared dilation margin \(m^2\) is varied to evaluate sensitivity on Swahili (SW, a representative low-resource language). Setting \(m^2=0\) removes dilation entirely, representing a pure attraction baseline:
| Configuration (\(m^2\)) | Operational Mechanism | PKU-SW Safety (%) โ | MultiJail-SW ASR (%) โ | Assessment |
|---|---|---|---|---|
| \(m^2 = 0.00\) (Shrink-only ablation) | Anchor attraction only, zero pairwise dilation | 93.89 | 2.86 | Over-compression degrades low-resource performance |
| \(m^2 = 0.15\) | Moderate pairwise repulsion margin | 96.11 | 1.90 | Steady safety recovery |
| \(m^2 = 0.30\) (Default) | Optimal hyperspherical shell equilibrium | 98.33 | 0.63 | Peak safety rate and lowest attack success rate |
| \(m^2 = 0.45\) | Excessive repulsion exceeding shell geometry capacity | 96.67 | 1.27 | Excessive repulsion conflicts with anchor attraction |
Key Findings¶
- Failure mode of pure attraction: The ablation at \(m^2=0\) empirically validates the danger of excessive collapse: without dilation, SW safety drops by 4.44% and MultiJail ASR increases more than fourfold compared to the balanced equilibrium at \(m^2=0.30\).
- Closing the low-resource vulnerability gap: Standard DPO yields high English safety (99.44%) but fails drastically on Swahili (7.22%), Pashto (30.00%), and Kurdish (17.22%). HEAL raises all three above 98% (reaching 100% on Kurdish), reducing cross-language variance by over three orders of magnitude (\(0.12440 \rightarrow 0.00010\)).
- Zero-shot out-of-distribution transfer: Without seeing any Italian, Javanese, Korean, Thai, or Vietnamese prompt-response data during training, HEAL achieves a 0.89% OOD jailbreak ASR on Qwen, demonstrating that the learned geometry transfers effectively across unseen language spaces.
- Preservation of general capabilities: HEAL incurs negligible impact on standard MMLU (73.09 vs. 73.10) and MMMLU-lite (60.50 vs. 60.37) benchmarks, confirming that safety enhancement does not come at the expense of general linguistic reasoning.
Highlights & Insights¶
- Asymmetric stop-gradient anchoring: Fixing the English representation as an immovable anchor provides an uncorrupted target for transfer, ensuring that noisy low-resource representations cannot pull the safety boundary away from its high-resource foundation.
- Geometric realization of alignment and uniformity: The paper provides a clear geometric solution to the trade-off by bounding both anchor attraction and pairwise repulsion within an analytically solvable shell manifold.
- Inference-free design: All auxiliary projections and geometric computations occur strictly during training; the projection head is removed for deployment, preserving original inference latency and memory footprints.
Limitations & Future Work¶
- Language set coverage: Evaluation is conducted across 10 in-distribution and 5 out-of-distribution languages; extending this formulation to dialectal variations, extremely low-resource languages, and code-switching scenarios remains unexplored.
- Backbone architecture scale: Empirical validations are centered on 7B to 9B parameter models; the geometric equilibrium behavior in massive models (70B+ or MoE architectures) warrants further investigation.
- Dependence on parallel prompt translation: The auxiliary loss requires parallel multilingual realizations of unsafe prompts during training, though it avoids the much harder problem of response-level supervision.
Related Work & Insights¶
- vs MLC (Multilingual Consistency): MLC forces rank-1 matrix collapse where the alignment direction is dynamically averaged across all languages, leading to noise contamination; HEAL anchors alignment onto English via stop-gradients and preserves representation spread via dilation.
- vs MPO (Multilingual Preference Optimization): MPO adjusts preference pairs across languages via reward gap estimation, which incurs higher supervision complexity; HEAL operates directly on the latent representation manifold, requiring no cross-lingual reward calibration.
- vs SDRRL (Self-Distillation): SDRRL relies on generating and translating responses from resource-rich languages to create distillation targets; HEAL sidesteps response generation and translation errors by regularizing prompt representations directly.
Rating¶
- Novelty: โญโญโญโญโ Elegantly adapts hyperspherical alignment-uniformity theory to multi-language preference learning via an anchored shell formulation.
- Experimental Thoroughness: โญโญโญโญโญ Comprehensive evaluations covering two backbones, in-distribution and out-of-distribution jailbreak attacks, margin sweeps, and general capability retention.
- Writing Quality: โญโญโญโญโญ Clear mathematical formulation, solid geometric intuition, and rigorous empirical validation.
- Value: โญโญโญโญโญ Offers a plug-and-play, computationally efficient auxiliary objective for community-wide multilingual LLM safety alignment.