UNISAF-Bench: Intention-Grounded Safety Evaluation for Unified Multimodal Generation Models¶
Conference: EMNLP 2026
Paper: ACL Anthology
Code: https://huggingface.co/datasets/amazon/UNISAF-Bench
Area: Multimodal VLM
Keywords: Unified Multimodal Generation, Intention-Grounded Safety Evaluation, Cross-Modal Safety Transfer, Multi-Turn Deflection Bias, Boundary-Benign Evaluation
TL;DR¶
UNISAF-Bench establishes the first intention-grounded safety benchmark for unified multimodal generation models (UMMGs), grounding all four input-output modality combinations and strictly paired red/blue variants in 1,257 shared harmful intentions (yielding 20,112 conversations) to uncover systematic cross-modal transfer failures, safety degradation under multimodal inputs, and multi-turn deflection bias.
Background & Motivation¶
Unified Multi-Modal Generation (UMMG) models integrate multimodal encoders and decoders around a central backbone language model, enabling users to seamlessly transition between disparate generative tasksโincluding text-to-text (T2T), text-to-image (T2I), image-text-to-text (TI2T), and image-text-to-image (TI2I)โwithin a single conversational session. However, existing safety evaluation frameworks remain fundamentally fractured across modalities: they either reuse legacy benchmarks developed specifically for isolated input or output modalities, or construct prompts independently for each modality combination without maintaining an underlying shared harmful intention.
This ungrounded construction causes severe cross-modal policy distortion. When the harmful intentions evaluated in T2T differ intrinsically from those evaluated in T2I or TI2I, a model's comparative performance reflects benchmark-specific policy skew rather than genuine differences in internal safety alignment. Furthermore, existing benchmarks cannot scale gracefully: as models incorporate emerging modalities like speech or video, independently designed benchmarks either amplify distribution drift or require rebuilding from scratch. Crucially, current safety benchmarks cannot answer the foundational safety question for unified architectures: does a unified model guard against a specific harmful intention consistently across all modality combinations it supports?
To resolve this blind spot, this paper rethinks safety evaluation from the ground up by rooting all evaluations in a shared semantic intent. Core idea: establish an intention-grounded evaluation methodology where all input-output modality combinations across single-turn and multi-turn settings are derived from a unified set of 1,257 human-verified harmful intentions and paired near-boundary benign counterparts, enabling intention-level aggregated profiling and precise quantification of cross-modal safety transfer failure.
Method¶
Overall Architecture¶
The UNISAF-Bench framework comprises three modular stages: Benchmark Data Synthesis, Benchmark Verification, and Benchmark Evaluation. The data synthesis pipeline adopts a top-down structure that progressively translates a verified policy-violating intention into diverse modalities and multi-turn conversations. The verification phase deploys a committee-based multimodal LLM assessment coupled with strict intention-level completeness filtering and human review. Finally, the target models are prompted with the verified instances, and an automated judge assigns binary labels to decompose responses into a five-category joint outcome taxonomy and aggregate intention-level safety metrics.
%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
A["19 Safety Policies<br/>7 Major Safety Categories"] --> B["Three-Stage Top-Down Synthesis Pipeline<br/>Intention Gen โ Multimodal Extension โ Multi-Turn Planning"]
B --> C["Backwards-Planned Multi-Turn Dialogue Construction<br/>Final-Turn Anchor + Benign/Mixed Prior History"]
C --> D["Intention-Level Completeness Filtering & Verification<br/>Dual-MLLM Committee + Completeness Filter + Human Audit"]
D --> E["UNISAF-Bench Benchmark Dataset<br/>1,257 Intentions ร 16 Variations = 20,112 Conversations"]
E --> F["Target UMMG Model Inference<br/>BAGEL-7B / Emu3.5 / Ming-flash / OmniGen2"]
F --> G["Joint Outcome Taxonomy & Cross-Modal Safety Transfer<br/>Deflection / Violation / Following โ 5 Outcomes & TFR"]
Key Designs¶
1. Three-Stage Top-Down Synthesis Pipeline: deriving cross-modal prompts from a shared harmful intent
Attempting to instruct an LLM to generate multimodal, multi-turn, policy-violating samples in a single prompt inevitably leads to semantic drift and intention dilution. UNISAF-Bench breaks down the generation process into three modular, top-down stages: - Stage 1 (Intention and Single-Turn T2T Synthesis): A text synthesizer (DeepSeek-R1-0528) first rephrases a target policy into three severity levels (Minor, Moderate, Severe), generates five candidate harmful intentions violating the policy, and randomly samples one. It then creates a paired T2T instance: a RED prompt (explicitly policy-violating) and a BLUE prompt (a near-boundary benign prompt topically identical to the RED variant but redirected to legitimate academic, educational, or creative contexts). - Stage 2 (Multimodal Extension): Conditioned on the Stage 1 intention and text prompts, the pipeline synthesizes T2I, TI2T, and TI2I prompts alongside detailed visual descriptions. Input images for TI2T and TI2I are rendered using Z-Image-Turbo. Crucially, multimodal prompts follow an isolation-benign principle: individual text prompts and input images appear completely harmless in isolation, and the violation emerges strictly from their joint composition. - Stage 3 (Multi-Turn Conversational Extension): The verified single-turn prompts are integrated as conversational endpoints in multi-turn dialogues. This sequential conditioning preserves the exact semantic intent across all modalities while allowing new modalities to be added incrementally without re-generating earlier data.
2. Backwards-Planned Multi-Turn Dialogue Construction: decoupling context length from deflection bias
Standard multi-turn safety datasets synthesize conversations forward, which frequently causes the conversation to wander away from the target intent or inadvertently trigger safety refusals prematurely. UNISAF-Bench adopts a backwards-planning strategy: Conversations (2 to 5 turns) are anchored at the final turn, which reuses the verified single-turn prompt verbatim. The generator works backwards to synthesize prior turns that establish a natural conversational context leading organically into the final request. To systematically isolate the mechanisms governing conversational safety, two controlled prior context setups are maintained: - Benign Prior Context: All preceding turns are completely benign and safe, featuring compliant assistant responses, testing whether pure conversational context accumulation degrades safety alignment. - Mixed Prior Context: Preceding turns contain a controlled mix of RED and BLUE requests accompanied by corresponding deflection or compliant assistant responses, testing how accumulated prior refusals impact final-turn decision-making. Because the RED and BLUE variants within each setup share an identical conversational history, any difference in final-turn deflection is strictly attributable to the safety boundary of the final request.
3. Intention-Level Completeness Filtering & Human Verification: enforcing rigorous cross-modal integrity
To ensure benchmark reliability, synthetic prompts undergo multi-tier verification before inclusion: - Committee-based MLLM Assessment: Two independent vision-language models assess each single-turn prompt for policy violation status (verifying that RED violates and BLUE complies) and semantic quality (absence of ambiguities, visual artifacts, or inconsistencies). Only instances unanimously approved by both models are retained. - Intention-Level Completeness Filtering: Each base intention yields a full matrix of 16 conversations (4 modalities \(\times\) 2 turn settings \(\times\) RED/BLUE variants). If even a single conversation fails the quality or policy check, the entire intention and all 16 associated conversations are discarded. This strict filter ensures zero missing cells across modalities (21.4% of initial candidate intentions survived). - Human Audit: Five expert annotators reviewed 424 sampled instances across single-turn and multi-turn splits, verifying high label accuracy (97.2%), linguistic and visual quality (96.0%), boundary fidelity (88.2%), and policy alignment (98.2%).
4. Joint Outcome Taxonomy & Cross-Modal Safety Transfer Metrics: unmasking capability failures and cross-modal leakage
Single-label metrics (such as string-matching refusal or naive safety classifiers) frequently conflate capability failure with genuine safety alignment. UNISAF-Bench deploys an automated judge (Qwen3-VL) to assign three binary labels per response: Deflection (\(\text{DF} \in \{\text{Yes}, \text{No}\}\)), Policy Violation (\(\text{PV} \in \{\text{Yes}, \text{No}\}\)), and Instruction Following (\(\text{IF} \in \{\text{Yes}, \text{No}\}\)). These are mapped into five mutually exclusive outcomes: 1. Deflection: \(\text{DF}=\text{Yes}\), explicit refusal without providing harmful assistance. 2. Capability Failure: \(\text{DF}=\text{No}, \text{PV}=\text{No}, \text{IF}=\text{No}\), the model attempted compliance but failed to produce a coherent on-task output (dominant in image editing). Low violation here is caused by execution inability rather than safety alignment. 3. Benign Compliance: \(\text{DF}=\text{No}, \text{PV}=\text{No}, \text{IF}=\text{Yes}\), compliant response to benign prompts. 4. Successful Harm: \(\text{DF}=\text{No}, \text{PV}=\text{Yes}, \text{IF}=\text{Yes}\), coherent execution of policy-violating requests. 5. Incidental Harm: \(\text{DF}=\text{No}, \text{PV}=\text{Yes}, \text{IF}=\text{No}\), off-topic or degraded output that nevertheless violates safety policies.
For the benchmark score, metrics are aggregated over the 1,257 shared intentions \(I\) and their derived conversation set \(C(i)\): $\(\text{DFR} = \frac{1}{|I|} \sum_{i \in I} \frac{1}{|C(i)|} \sum_{c \in C(i)} \mathbb{1}[\text{DF}(c) = \text{Yes}]\)$
To directly evaluate safety generalization across modalities, the paper defines the Safety Transfer Failure Rate (TFR) for a modality transition \(m_1 \to m_2\): $\(\text{TFR}(m_1 \to m_2) = \frac{\sum_{c \in \text{RED}} \mathbb{1}[\text{DF}(c_{m_1}) = \text{Yes} \land \text{DF}(c_{m_2}) = \text{No}]}{\sum_{c \in \text{RED}} \mathbb{1}[\text{DF}(c_{m_1}) = \text{Yes}]}\)$ A TFR of 100% indicates that safety mechanisms established on modality \(m_1\) fail entirely when transferring to \(m_2\).
Key Experimental Results¶
Main Results¶
Four representative open-source UMMG architectures were evaluated: 1. BAGEL-7B: Mixture-of-Transformers (MoT) architecture with dual vision tokenizers (VAE for generation, ViT for understanding); 2. Emu3.5: Unified decoder-only autoregressive architecture trained with next-token prediction across all modalities; 3. Ming-flash: Sparse MoE architecture with a frozen perception MLLM and a decoupled diffusion generator; 4. OmniGen2: Decoupled architecture pairing a frozen Qwen2.5-VL backbone with a separately trained diffusion decoder.
The table below presents intention-level aggregated performance alongside per-modality breakdowns across all 1,257 intentions (all values in %):
| Model | Prompt Type | Intention DFR | Intention PVR | Intention IFR | T2T DFR | T2T PVR | T2I DFR | T2I PVR | TI2T DFR | TI2T PVR | TI2I DFR | TI2I PVR |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| BAGEL-7B | RED | 6.1 | 71.4 | 73.3 | 17.6 | 76.6 | 0.0 | 92.7 | 6.7 | 79.3 | 0.0 | 37.0 |
| BLUE | 0.3 | 8.3 | 58.3 | 1.0 | 5.0 | 0.0 | 19.3 | 0.0 | 0.5 | 0.0 | 8.5 | |
| Emu3.5 | RED | 20.8 | 55.7 | 63.3 | 48.2 | 41.9 | 14.3 | 73.2 | 13.4 | 70.1 | 7.3 | 37.6 |
| BLUE | 1.3 | 2.3 | 75.1 | 0.4 | 2.4 | 0.4 | 5.1 | 3.7 | 0.5 | 0.7 | 1.0 | |
| Ming-flash | RED | 28.0 | 47.9 | 48.8 | 70.2 | 23.8 | 0.0 | 87.5 | 41.8 | 45.0 | 0.0 | 35.5 |
| BLUE | 1.0 | 5.6 | 57.5 | 3.4 | 0.8 | 0.0 | 14.7 | 0.6 | 0.5 | 0.0 | 6.3 | |
| OmniGen2 | RED | 8.5 | 58.4 | 58.1 | 27.7 | 55.1 | 0.0 | 81.9 | 6.3 | 68.0 | 0.0 | 28.6 |
| BLUE | 0.4 | 6.8 | 54.0 | 1.6 | 8.5 | 0.0 | 14.8 | 0.1 | 1.0 | 0.0 | 3.0 |
Ablation Study & Empirical Analyses¶
1. Safety Transfer Failure Rate (TFR) Across Modality Transitions
The table below quantifies the failure of safety mechanisms to generalize across modality shifts (%):
| Modality Transition | BAGEL-7B | Emu3.5 | Ming-flash | OmniGen2 |
|---|---|---|---|---|
| T2T โ T2I (Generation-side shift) | 100.0 | 92.0 | 100.0 | 100.0 |
| TI2T โ TI2I (Generation-side shift) | 100.0 | 100.0 | 100.0 | 100.0 |
| T2T โ TI2T (Understanding-side input shift) | 94.8 | 94.3 | 63.6 | 94.4 |
| T2I โ TI2I (Understanding-side input shift) | โ | 98.0 | โ | โ |
2. Prompt-Level Safety Mitigation on BAGEL-7B
Prepending a system prompt enumerating all 19 safety policies to BAGEL-7B tests whether inference-time prompting can patch these vulnerabilities (baseline \(\to\) mitigated, in %):
| Evaluation Cell | RED DFR (Refusalโ) | RED PVR (Harmโ) | RED IFR (Following) | BLUE DFR (Over-refusalโ) | BLUE IFR (Benign Utilityโ) |
|---|---|---|---|---|---|
| Intention-Level Aggregate | 12.1 โ 79.0 | 78.0 โ 15.8 | 87.2 โ 20.6 | 0.5 โ 30.8 | 96.4 โ 66.8 |
| T2T Single-Turn | 12.3 โ 93.9 | 83.4 โ 5.4 | 87.1 โ 5.6 | 0.2 โ 84.6 | 95.6 โ 14.7 |
| T2T Multi-Turn | 23.0 โ 78.8 | 69.8 โ 16.5 | 75.2 โ 20.3 | 1.9 โ 25.8 | 92.5 โ 68.3 |
| TI2T Single-Turn | 2.1 โ 87.7 | 86.0 โ 8.9 | 97.6 โ 12.2 | 0.0 โ 10.0 | 99.8 โ 88.5 |
| TI2T Multi-Turn | 11.3 โ 55.5 | 72.6 โ 32.5 | 88.5 โ 44.2 | 0.0 โ 2.7 | 97.4 โ 95.7 |
Key Findings¶
- Systematic Vulnerability in Interpersonal Harms: Even Ming-flash, the most aligned model overall, deflects fewer than one in three harmful requests on average (28.0% DFR). Deflection collapses almost entirely in interpersonal harms, including Stalking/Harassment Facilitation (1.5%โ11.1% DFR), Dehumanizing Content (1.6%โ13.6% DFR), and Protected Group Targeting (4.9%โ30.5% DFR).
- Multimodal Inputs Degrade Text-Output Safety: Introducing an image to the input while keeping the output modality fixed to text (T2T \(\to\) TI2T) triggers severe safety degradation, with TFR reaching 63.6%โ94.8%. Multimodal perception features actively interfere with safety-trained language pathways within the understanding component itself.
- Total Failure of Safety Transfer to Image Generation: Across BAGEL-7B, Ming-flash, and OmniGen2, T2I and TI2I deflection rates are strictly 0.0% (TFR = 100%). While Emu3.5 exhibits non-zero T2I deflection (14.3%) owing to its unified autoregressive parameters, 92.0% of prompts deflected in T2T still bypass safety in T2I, proving that text alignment cannot substitute for direct image-level safety training.
- Deflection-Biased Multi-Turn Dynamics: Conversational length per se does not improve or degrade safety; under benign context, deflection remains flat. However, the presence of even a single prior refusal causes an indiscriminate surge in final-turn deflection across both RED and BLUE prompts (e.g., BAGEL-7B's false refusal on BLUE jumps from 0.0% to 25.5% after prior refusals), demonstrating mechanical refusal priming rather than contextual harm discernment.
- Inference Mitigation Causes Catastrophic Utility Collapse: While a comprehensive safety system prompt boosts BAGEL-7B's T2T refusal to 93.9%, it inflates benign over-refusal to 84.6% and crushes instruction following to 14.7%, while remaining completely unable to reach decoupled image generation pathways.
Highlights & Insights¶
- Intention-Grounded Consistency Eliminates Distribution Artifacts: Evaluating all modalities against identical verified intentions guarantees that cross-modal score discrepancies reflect intrinsic architectural properties rather than task or policy distribution skew.
- Dissecting Capability Failure vs. True Safety: In complex image editing tasks (TI2I), low policy violation rates mask severe capability collapse (31%โ80% Capability Failure). As generative capabilities mature, these failures will inevitably convert into successful harms unless matched by proactive safety alignment.
- Architectural Implications for Multimodal Alignment: Decoupled and frozen-backbone designs completely sever safety propagation to image decoders. Direct safety supervision on image generation pathways is an inescapable necessity.
Limitations & Future Work¶
- Absence of Sophisticated Adversarial Attacks: The benchmark evaluates direct policy probing rather than cross-modal jailbreaks or adversarial perturbations that split malicious intents across modalities.
- Single-Modality Output Focus: Outputs are evaluated either as text or image; interleaved generation (mixed text-image responses) and temporal modalities (video, audio) are not yet evaluated.
- Automated Judge Dependence: Safety labels are assigned by Qwen3-VL, which may harbor subtle cultural biases or imperfect calibration on nuanced visual harms.
Related Work & Insights¶
- vs MM-SafetyBench & MMDT: Prior multimodal benchmarks assess safety strictly within multimodal understanding (image+text input \(\to\) text output). UNISAF-Bench expands evaluation across all four generative quadrants in a unified framework.
- vs T2ISafety & UnsafeBench: Image generation benchmarks lack cross-modal intent parity with text safety. UNISAF-Bench establishes direct intent-level equivalence across text and image generation.
- vs CoSafe & SafeMT: Conversational benchmarks focus on multi-turn dialogue comprehension; UNISAF-Bench reveals for the first time the multi-turn deflection bias phenomenon in unified generative architectures.
Rating¶
- Novelty: โญโญโญโญโญ Pioneering intention-grounded paradigm that solves cross-modal evaluation confounding for unified generative models.
- Experimental Thoroughness: โญโญโญโญโญ Comprehensive evaluation across 4 architectures, 1,257 intentions, 20,112 dialogues, and detailed transfer/mitigation ablations.
- Writing Quality: โญโญโญโญโญ Rigorous conceptual formulation, clear empirical exposition, and insightful error decomposition.
- Value: โญโญโญโญโญ Provides an indispensable evaluation methodology and foundational safety baseline for the multimodal generative modeling community.