Q-Align: Query-Boundary Multilingual Alignment for LLM Safety Guardrails¶
Conference: EMNLP 2026
Paper: ACL Anthology
Code: https://github.com/Alibaba-AAIG/Q-Align
Area: LLM Safety
Keywords: Multilingual Safety Guardrails, Cross-Lingual Representation Alignment, Query-Boundary Representation, Confidence-Weighted Anchor, Generative Safety Moderation
TL;DR¶
Q-Align resolves the severe degradation of multilingual query-response safety detection caused by conventional last-token alignment, achieving consistent gains across both pre-generation and post-generation safety barriers via query-boundary alignment and confidence-weighted anchors.
Background & Motivation¶
The global deployment of large language models (LLMs) requires robust multilingual safety guardrails to moderate both incoming user prompts and generated model responses. In practical production pipelines, generative safety guardrails serve two distinct protective roles: pre-generation risk detection on user inputs alone (the Query-only scenario, Q) and post-generation evaluation of whether the assistant's reply complies with safety policies or properly refuses harmful requests (the Query-Response scenario, QR). However, existing open guardrail models (such as LlamaGuard, XGuard, and PolyGuard) exhibit severely deficient moderation capabilities in low-resource languages. Prior remedies typically rely on synthesizing massive multilingual parallel corpora via machine translation, which incurs prohibitive computational overhead and frequently fails to generalize to unseen, out-of-domain languages.
Cross-lingual representation transfer offers a promising, sample-efficient alternative by pulling semantically equivalent inputs into a shared semantic subspace. Recent findings demonstrate that multilingual LLMs capture shared semantics in intermediate and deep layers, suggesting that aligning hidden representations on limited parallel data can facilitate knowledge transfer from high-resource to low-resource languages. Nevertheless, this paper identifies a critical failure mode: while aligning hidden states at the very last token of the sequence improves query-only (Q) risk detection, it consistently and severely degrades multilingual query-response (QR) safety evaluation. For instance, standard fine-tuning and last-token alignment degrade XGuard's QR F1 score on the RTP-LX benchmark from 88.40% to 79.41%. The root cause lies in the fact that response safety evaluation is strictly conditioned on the query intent; forcing representations at the end of the full sequence to converge conflates shared query intent with language-specific pragmatic response cues (e.g., tone of refusal, indirect politeness, and localized syntactic markers), effectively stripping away the discriminative features required for post-generation response safety evaluation.
The paper's angle of attack is to decouple user intent alignment from language-specific response evaluation at their natural temporal junction. Because the user's intent is fully established once the query finishes, whereas evaluating the subsequent response relies strictly on understanding that prior intent, semantic alignment should happen precisely at the query boundary before response tokens are attended to. Core idea: align multilingual hidden representations strictly at the query-boundary token toward a dynamic confidence-weighted semantic anchor and train within typologically coherent language groups, thereby standardizing cross-lingual intent comprehension while fully preserving language-specific response evaluation cues.
Method¶
Overall Architecture¶
Q-Align takes parallel multilingual safety samples as input, where each instance consists of semantically equivalent inputs across \(K\) languages, formatted with a shared system prompt, a user query, and a candidate assistant response. In a single forward pass, the model calculates the standard next-token cross-entropy loss for each language and extracts the hidden state from a deep residual stream layer at the exact query-boundary token. The model then dynamically constructs a per-sample alignment anchor by weighting each language's boundary representation according to its prediction confidence (inversely related to its cross-entropy loss). Finally, a mean squared error alignment loss pulls each language's query representation toward the dynamic anchor within genealogically structured language groups, jointly optimizing the guardrail for task supervision and semantic alignment.
%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
A["Multilingual Parallel Input<br/>System Prompt + Query + Response"] --> B["Query-Boundary Hidden State Extraction<br/>Extract layer-31 residual state at boundary token t*"]
B --> C["Confidence-Weighted Dynamic Anchor<br/>Compute softmax weights from per-language CE loss"]
C --> D["Genealogical Grouped Alignment Training<br/>Co-train related language groups + English anchor + MSE loss"]
D --> E["Generative Guardrail Output<br/>Decoupled intent transfer with preserved response cues"]
Key Designs¶
1. Query-Boundary Hidden State Extraction: Decoupling Query Intent from Response Evaluation Conventional cross-lingual alignment forces representation convergence at the final sequence token, ignoring the intrinsic conditional factorization of query-response safety assessment: \(p_\theta(y_r \mid sp, q, r) = p_\theta(y_r \mid \phi_\theta(sp, q), r)\). Under this formulation, judging the safety of response \(r\) requires an accurate internal encoding of the query context \(\phi_\theta(sp, q)\), combined with the unadulterated linguistic cues contained within \(r\) itself. Aligning at the final token forces language-specific response features into an artificial cross-lingual average, destroying the nuanced pragmatic signals needed to distinguish safe refusals from subtle violations. Q-Align restricts alignment strictly to the query boundary position \(t^* = |sp| + |q|\), where the system prompt and user query are completely encoded but before any response token is processed. By extracting the hidden state \(h^{(i)}_k = h^{(\ell)}_{t^*}(x^{(i)}_k)\) from layer \(\ell\) (e.g., layer 31 of an 8B model), the alignment objective regularizes only the query intent representation across languages, leaving the response evaluation pathway free from cross-lingual distortion.
2. Confidence-Weighted Dynamic Anchor: Guiding Transfer by Reliable Language Predictions Standard alignment techniques typically pull representations toward an unweighted centroid or a fixed English representation. However, a fixed English anchor assumes English is always the most accurate target, whereas uniform averaging allows low-quality representations from poorly understood languages to corrupt the anchor. Q-Align establishes a sample-adaptive dynamic anchor based on model confidence. During training on parallel sample \(S^{(i)}\), each language \(k\) produces a token-level cross-entropy loss \(L^{(i,k)}_{CE}\). A lower cross-entropy indicates that the model already comprehends the intent and safety label reliably in that language. Q-Align computes a normalized weight distribution using a softmax function with temperature \(\tau > 0\):
The sample's alignment anchor is formed as the confidence-weighted combination \(h^{*(i)} = \sum_{k=1}^K \alpha^{(i)}_k h^{(i)}_k\). This ensures that languages demonstrating superior intent comprehension dominate the anchor representation, pulling less reliable languages toward a trustworthy semantic center while smoothly transitioning toward a balanced multilingual consensus as training progresses.
3. Geographically and Genealogically Guided Grouping: Minimizing Representation Conflict Aligning all typologically diverse languages within a single unconstrained pool induces conflicting gradient updates, as distant language families naturally occupy disparate representation subspaces. Q-Align organizes languages into coherent alignment groups based on genealogical and geographical affinity according to three design principles. First, genealogically related languages (e.g., East Asian: Chinese, Japanese, Korean, Thai; Romance + Indo-Aryan; Germanic; Slavic) are co-trained within the same group to exploit natural representation clustering. Second, English is incorporated into every group as a high-resource, low-entropy reference anchor. Third, isolated languages lacking close relatives in the dataset (e.g., Arabic from Afro-Asiatic, Swahili from Niger-Congo, and Javanese from Austronesian) are evenly distributed across groups with strong cores, allowing them to benefit from a coherent alignment signal without clustering weak languages together.
Loss & Training¶
The alignment objective measures the mean squared error between each language's boundary representation and the confidence-weighted anchor:
The overall training objective combines the autoregressive cross-entropy loss with the weighted alignment loss:
Hyperparameters are configured with anchor temperature \(\tau = 0.5\), boundary extraction placed at layer 31, and alignment weight \(\lambda\) scaling exponentially with \(\lambda_{max} = 0.3\) and decay rate \(\gamma = 0.1\). Training is conducted on 4 NVIDIA L20 GPUs with an effective batch size of 128 over approximately 4,800 parallel samples, converging in just 0.5 hours of wall-clock time.
Key Experimental Results¶
Main Results¶
The models were evaluated on the 10-language MultiJail benchmark and the comprehensive RTP-LX benchmark spanning 39 languages for Query-only (Q) and 27 languages for Query-Response (QR) scenarios. Experiments were conducted across three 8B guardrail architectures: XGuard, Qwen3Guard, and PolyGuard-Qwen, using macro F1 score (%) as the primary metric.
| Benchmark / Scenario | Backbone Model | Baseline | SFT | MLC | Q-Align (Ours) | Gain over Baseline |
|---|---|---|---|---|---|---|
| MultiJail (Q Scenario) | XGuard (8B) | 90.30 | 90.85 | 88.16 | 92.76 | +2.46 |
| MultiJail (Q Scenario) | Qwen3Guard (Strict) | 90.65 | 91.32 | 96.57 | 94.31 | +3.66 |
| MultiJail (Q Scenario) | PolyGuard-Qwen | 85.14 | 88.30 | 92.27 | 95.00 | +9.86 |
| RTP-LX (Q Scenario) | XGuard (8B) | 82.10 | 86.82 | 81.86 | 87.21 | +5.11 |
| RTP-LX (Q Scenario) | PolyGuard-Qwen | 80.92 | 83.49 | 83.86 | 84.92 | +4.00 |
| RTP-LX (QR Scenario) | XGuard (8B) | 88.40 | 79.41 | 78.87 | 90.93 | +2.53 (+11.52 vs SFT) |
| RTP-LX (QR Scenario) | PolyGuard-Qwen | 74.70 | 72.30 | 76.25 | 80.79 | +6.09 (+8.49 vs SFT) |
Gains are particularly remarkable on low-resource languages. On MultiJail, PolyGuard-Qwen improves from 32.45% to 83.55% (+51.10 F1) on Swahili (SW) and from 72.47% to 91.94% (+19.47 F1) on Javanese (JV).
Ablation Study¶
Ablations on alignment positions, anchor weighting strategies, and grouping schemes on the XGuard backbone demonstrate the critical contribution of each component.
| Ablation Dimension | Configuration | RTP-LX (Q) | RTP-LX (QR) | Key Analytical Takeaway |
|---|---|---|---|---|
| Alignment Position | Baseline (Unaligned) | 82.10 | 88.40 | Default unaligned baseline |
| Alignment Position | Final Token Alignment (QR-Align) | 84.27 | 87.42 | Improves Q but harms QR, confirming signal dilution |
| Alignment Position | Query-Boundary Alignment (Q-Align) | 87.21 | 90.93 | Concurrently maximizes F1 across both Q and QR |
| Anchor Formulation | Uniform averaging | 85.85 | 89.35 | Unweighted average is degraded by noisy languages |
| Anchor Formulation | Fixed English anchor | 85.56 | 88.57 | Weakest anchor; English is not universally superior |
| Anchor Formulation | Random weighting | 85.87 | 89.27 | Lacks confidence guidance; unstable transfer |
| Anchor Formulation | Single lowest-loss language (Simple-centroid) | 86.40 | 89.32 | Discards multilingual diversity, harming QR |
| Anchor Formulation | Confidence-weighted anchor (Ours) | 87.21 | 90.93 | Optimal balance between reliability and diversity |
| Grouping Strategy | Random language assignment | 82.51 | 83.27 | Conflicting typological gradients severely drop QR |
| Grouping Strategy | Genealogical & Geographical (Q-Align) | 86.53 | 90.84 | Coherent subspace alignment yields +7.57 F1 in QR |
Key Findings¶
- Resolution of QR Safety Collapse: While data SFT and last-token alignment (MLC/QR-Align) provide modest improvements in Q scenarios, they trigger severe regression in multilingual QR scenarios (dropping XGuard from 88.40% to 79.41% on RTP-LX). Q-Align completely reverses this decline by confining alignment to the query boundary, establishing that query comprehension is a necessary prerequisite for QR judgment while response representations must remain unconstrained.
- Robust Out-of-Distribution (OOD) Generalization: On languages completely absent from training data (including Bulgarian, Welsh, Estonian, and Greek), Q-Align delivers steady gains over the baseline (+4.97 F1 on Q, +4.17 F1 on QR on XGuard). Even on the zero-resource Welsh language (CY), Q-Align achieves a +2.52 F1 improvement without any direct supervision, validating genuine cross-lingual subspace convergence.
- Preservation of English Safety Capabilities: Across five standard English-only safety benchmarks (Aegis_2.0, S_Eval, SimpleSafetyTests, ToxicChat, and SafeRLHF), Q-Align avoids catastrophic forgetting and even slightly boosts performance (improving average English QR F1 on XGuard from 88.87% to 91.24%), proving that query-boundary alignment reinforces the backbone's core decision boundaries.
Highlights & Insights¶
- Surgical Decoupling of Temporal Safety Semantics: The key insight is recognizing the causal asymmetry in guardrail processing: while the query sets the safety context, the response carries localized pragmatic nuance. Pinpointing the alignment position at the exact query boundary preserves downstream response discrimination.
- Confidence-Weighted Dynamic Anchor Guidance: By mapping per-sample token-level cross-entropy through a softmax function, the anchor dynamically allows the language with the highest predictive confidence to guide less familiar languages without discarding semantic diversity.
- Remarkable Data and Compute Efficiency: Rather than synthesizing millions of samples (such as PolyGuard's 1.91M dataset), Q-Align attains superior cross-lingual transfer using only 4,800 parallel samples translated from 1,200 seeds, completing alignment in 30 minutes on four L20 GPUs and transferring successfully to lightweight 0.6B models.
Limitations & Future Work¶
- Author-Acknowledged Limitations: The empirical study evaluates models up to 8B parameters across 17 training languages. Testing on larger 70B+ architectures and more distant indigenous language families remains for future work. Additionally, the alignment layer was empirically chosen at layer 31; adapting to new base architectures may require minor layer search.
- Future Directions: Exploring dynamic query-boundary extraction in multi-turn dialogues where conversational context continuously shifts, as well as extending query-boundary alignment to multimodal vision-language guardrails (e.g., aligning visual-text prompt boundaries).
Related Work & Insights¶
- vs Large-Scale Data Synthesis (PolyGuard, DuoGuard): PolyGuard requires nearly 2 million synthetic samples, and DuoGuard relies on complex two-player RL data synthesis. Q-Align demonstrates that alignment in the latent representation space is far more data-efficient and provides superior generalization to unseen languages.
- vs Full-Sequence Representation Alignment (MLC, LASA): MLC enforces consistency at the sequence tail, and LASA trains an interpreter at an intermediate bottleneck. Q-Align proves that tail-token alignment inherently degrades multilingual QR safety, establishing query-boundary alignment as the correct paradigm for generative guardrails.
Rating¶
- Novelty: โญโญโญโญโญ Decouples query intent from response evaluation via query-boundary alignment and confidence-weighted anchors, solving a genuine failure mode in multilingual guardrails.
- Experimental Thoroughness: โญโญโญโญโญ Rigorously tested across 3 guardrail backbones (plus a 0.6B model), 10 to 39 languages across Q and QR scenarios, unseen language transfer, English retention, and layer-wise CKA analysis.
- Writing Quality: โญโญโญโญโญ Exceptionally clear mathematical formulation, compelling motivation, and transparent presentation of trade-offs.
- Value: โญโญโญโญโญ Delivers an immediately actionable, compute-efficient recipe for production-grade multilingual AI safety systems.