MAViQ: Memory-Augmented Visual Querying for Multi-Turn MLLM Jailbreaks¶
Conference: EMNLP 2026
Paper: ACL Anthology
Code: https://github.com/jixingtian/Maviq
Area: Large Language Model Reasoning
Keywords: Multimodal LLMs, Multi-Turn Jailbreak, Trace-to-Query Realization, Dual-Level Memory, Image-Grounded Querying
TL;DR¶
MAViQ formalizes multi-turn multimodal jailbreaking into a trace-to-query realization paradigm, decomposing dialogue feedback into explicit textual intent, visual roles, and cross-modal handoffs backed by dual-level memory and post-generation image grounding, surpassing the strongest baseline by 5.90% and 8.32% ASR across six target MLLMs.
Background & Motivation¶
As multimodal large language models (MLLMs) rapidly expand from basic captioning into complex document processing, multi-image comparative analysis, interface navigation, and embodied vision-language reasoning, their vulnerability to adversarial safety exploitation has become a pressing security concern. Early multimodal jailbreaks focused primarily on static single-turn prompt crafting. Common techniques include encoding banned instructions into typographic images (FigStep), splitting unsafe concepts across visual and textual channels (Jailbreak in Pieces), exploiting shuffle-induced cross-modal inconsistency (SI-Attack), or staging elaborate visual role-play backdrops (VRP). However, modern production MLLMs equipped with alignment guardrails readily detect such one-shot, pre-constructed adversarial inputs. This resilience forces attackers toward interactive multi-turn strategies where cumulative dialogue turns gradually probe and weaken safety boundaries.
Despite the promise of multi-turn interactions, current multimodal jailbreak paradigms suffer from a structural disconnect in how target feedback is converted into subsequent queries. On one hand, candidate-search attacks (such as VisCo and IDEATOR) explore diverse image-text inputs, but their search units remain static pre-constructed candidates rather than adaptive, feedback-shaped queries. On the other hand, established text-only multi-turn attacks (such as Crescendo and FITD) demonstrate that dialogue responses can effectively guide subsequent probing, yet their progressions remain entirely textual. In vision-language models, an accumulated clue from a prior response cannot simply be appended as text; it must be systematically assigned to textual intent, visual entities/actions, or cross-modal referencing phrases. Without explicit modality-aware allocation, multi-turn attacks inevitably degenerate into text-only escalation decorated with uninformative images, or they cause severe image-text mismatch that triggers immediate defensive refusals.
The fundamental tension in multi-turn multimodal jailbreaking lies in this realization gap: while the target model observes only the target-visible dialogue context alongside the concrete image-text pair, the attacker must transform an evolving history of partial concessions, refused probes, and exploratory avenues into a coherent, visually grounded next-turn query that remains natural and stealthy. Core idea: formalize multi-turn multimodal jailbreaks as trace-to-query realization via MAViQ, leveraging dual-level trajectory-scenario memory to explicitly allocate textual, visual, and handoff roles, followed by post-generation visual inspection to ensure the final query strictly grounds in visible image evidence.
Method¶
Overall Architecture¶
MAViQ formulates the jailbreak pipeline as an iterative, black-box trace-to-query realization framework structured into four attacker-side stages coupled with a feedback routing control loop. Given a harmful target goal \(g\) and a safety scenario type \(c\), the system cycles through the following pipeline at each turn \(t\): first, a dual-level memory system supplies instance-specific interaction traces (trajectory memory) and cross-instance tactical priors (scenario memory); second, the Memory-Augmented Modality Allocator translates accumulated clues into a structured modality split specifying textual intent, visual roles, cross-modal handoffs, and an image generation prompt; third, a text-to-image diffusion model synthesizes the raw image, after which the Image-Grounded Query Realizer inspects the rendered visual content before composing the final target-facing question with a caption panel; fourth, the target MLLM response is evaluated by an LLM-based feedback router that determines success, triggers rewrite retries on refusal, or routes progress and stalled states directly into memory updates.
%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
A["Harmful Goal & Scenario Input<br/>Attack Initialization"] --> B["Dual-Level Memory Retrieval<br/>Dual-Level Memory"]
B --> C["Modality Planning & Splitting<br/>Memory-Augmented Modality Allocator"]
C --> D["Diffusion Synthesis & Grounding<br/>Image-Grounded Query Realizer"]
D --> E["Target Invocation & Outcome Routing<br/>Feedback Router & Memory Updater"]
E -->|Refusal / Rewrite Branch| C
E -->|Success / Maximum Budget| F["Final Attack Output & Scenario Prior<br/>Scenario Knowledge Base"]
Key Designs¶
1. Dual-Level Memory: resolving the trade-off between cross-instance transfer and instance-specific adaptation
Static attacks discard historical interaction data entirely, while naive multi-turn systems risk polluting the target-facing dialogue if planning history is exposed. MAViQ introduces a clean separation between two memory tiers. Local Trajectory Memory \(\mathcal{M}_{\text{traj}}^t = (C_t^+, C_t^-, \sigma_t)\) tracks real-time dynamics within the active attack instance: \(C_t^+\) stores target-accepted framings, verified technical terminology, and functional visual references; \(C_t^-\) logs refused phrasing, defensive redirects, and failed cross-modal carriers; and \(\sigma_t\) summarizes the active state. In contrast, global Scenario Memory \(\mathcal{B}_c\) archives reusable multi-turn patterns across completed attacks within the same safety category \(c\). Each scenario record \(b_j = (\pi_j, \rho_j, \gamma_j, \alpha_j)\) encapsulates a high-level framing pattern \(\pi_j\), an ordered trajectory strategy \(\rho_j\), a visual-anchor constraint \(\gamma_j\), and an empirical success score \(\alpha_j\). In the initial turn (\(t=1\)), MAViQ retrieves the top-3 scoring scenario records to seed planning; in subsequent turns (\(t>1\)), the allocator switches entirely to local trajectory memory and active context, allowing real-time refusals and target concessions to override initial priors.
2. Memory-Augmented Modality Allocator: preventing modality mismatch through explicit role allocation
When target feedback yields exploitable clues, the attacker must decide which clues remain in text, which migrate to visual evidence, and how the modalities reference each other. Operating prior to image generation, the allocator takes the goal, visible dialogue history, routing state, and memory traces to synthesize a structured next-turn plan \(z_t = (s_t, f_t, a_t^T, a_t^V, h_t, p_t^I)\). Here, \(s_t\) defines the intermediate subgoal, \(f_t\) selects an overarching framing style (hypothetical, fictional, or academic), \(a_t^T\) specifies the explicit textual question intent, \(a_t^V\) determines the concrete objects, physical actions, or spatial configurations to be visually rendered, \(h_t\) provides a generic cross-modal handoff phrase, and \(p_t^I\) contains a concise image generation prompt. By decoupling visual roles from textual inquiries before querying the target, this design ensures that the generated image provides genuine physical context rather than acting as a redundant visual decoration.
3. Image-Grounded Query Realizer: eliminating text-image hallucination and phantom visual references
Text-to-image diffusion models (e.g., Stable Diffusion 3.5 Large) frequently fail to render fine-grained details or adhere strictly to complex prompt constraints. If the final text query is composed prior to image synthesis, it may reference visual artifacts that do not actually exist in the generated picture, prompting the target MLLM to dismiss the inquiry with a simple "the requested item is not visible in the image." To close this loop, MAViQ introduces an image-grounded realization stage: the diffusion model \(G\) first synthesizes a raw image \(I_t^{\text{raw}}\), which is then inspected by the Query Realizer \(P_{\text{real}}\) alongside the planned intent \(z_t\) and context \(\tilde{H}_t\). The realizer references visual cues only if they are genuinely supported by the rendered scene; if a planned visual anchor is missing, the query smoothly falls back to textual continuity without hallucinating image features. Furthermore, a short visual-anchor caption panel derived deterministically from the prompt is appended beneath the image to form the final target input \(I_t\), stabilizing the target model's perceptual parsing of subtle entities.
4. Feedback Router & Memory Updater: transforming black-box defensive signals into cumulative attack momentum
Responses from the black-box target model must be systematically parsed to steer subsequent attack iterations. MAViQ employs a tiered evaluation cascade: an LLM-based success judge \(J_{\text{succ}}\) first determines whether the response provides actionable, substantive assistance toward the core harmful goal (scoring 10/10 without disclaimers); if unsuccessful, a refusal detector \(J_{\text{ref}}\) checks whether the response constitutes an explicit refusal or defensive block relative to the active subgoal. If refused, the turn outcome is marked as rewrite, the rewrite counter \(c_{\text{rew}}\) is incremented, the failed probe is recorded into \(C_t^-\) as negative evidence, and the refused exchange is purged from the target-facing context \(\tilde{H}_t\). Non-refusals pass to the progress router \(J_{\text{prog}}\), which assigns a goal-proximity score to label the turn as progress (advancing toward harmful deliverables) or stalled (benign drift requiring a carrier pivot). Upon session completion, the full trajectory is compressed into a candidate scenario record scoring \(u\), updating matching records via an exponential moving average:
or inserting a new scenario prior when no close match exists (\(\lambda = 0.3\)), creating a continuously evolving repository of adversarial priors.
Benchmark & Evaluation Protocol¶
Experiments are evaluated on two multimodal safety benchmarks: MM-SafetyBench (1,680 instances across 13 prohibited scenarios) and JailbreakV-mini (280 instances across diverse safety categories). Six frontier target MLLMs are evaluated under a strict black-box setup: open-source models Qwen3-VL-8B-Instruct, LLaVA-OneVision-1.5-8B-Instruct, and InternVL-2.5-8B; alongside production closed-source models GPT-4o, GPT-5.1, and Gemini-3.1-Pro-Preview. The default operational budgets are set to an Effective-Turn Budget of \(T=5\) and an equal Rewrite Budget of \(R=5\) (capping total target invocations at 10 Target Calls per instance). Attacker-side auxiliary modules are powered by GPT-4o, image generation uses Stable Diffusion 3.5 Large, and safety outcomes are scored by a GPT-5.1 judge under a rigorous 1–10 rubric where only substantive violations rated 10 are counted as successful jailbreaks.
Key Experimental Results¶
Main Results¶
Under a unified evaluation protocol and budget accounting, MAViQ was evaluated against six established multimodal jailbreak baselines spanning typographic prompts (FigStep), compositional intent splitting (Jailbreak in Pieces), shuffle inconsistency (SI-Attack), safe visual composition (SSA), visual context injection (VisCo), and multi-agent search (IDEATOR).
| Benchmark | Method | Qwen3VL | LLaVA-OV-1.5 | InternVL-2.5 | GPT-4o | GPT-5.1 | Gemini-3.1-Pro | Average ASR (%) |
|---|---|---|---|---|---|---|---|---|
| MM-SafetyBench | FigStep | 34.70 | 45.95 | 42.56 | 35.18 | 25.00 | 26.13 | 34.92 |
| MM-SafetyBench | SI-Attack | 12.02 | 20.48 | 17.98 | 13.99 | 2.02 | 6.73 | 12.20 |
| MM-SafetyBench | SSA | 41.37 | 50.71 | 48.39 | 46.19 | 15.95 | 20.83 | 37.24 |
| MM-SafetyBench | VRP | 27.98 | 51.96 | 42.38 | 31.96 | 8.33 | 15.77 | 29.73 |
| MM-SafetyBench | VisCo | 56.55 | 61.19 | 58.99 | 57.50 | 47.68 | 50.95 | 55.48 |
| MM-SafetyBench | IDEATOR | 66.79 | 74.23 | 70.77 | 67.98 | 53.98 | 54.46 | 64.70 |
| MM-SafetyBench | MAViQ (Ours) | 70.36 | 75.77 | 72.98 | 72.68 | 66.37 | 65.42 | 70.60 |
| JailbreakV-mini | FigStep | 52.86 | 68.21 | 62.50 | 55.36 | 40.00 | 50.36 | 54.88 |
| JailbreakV-mini | SI-Attack | 14.64 | 42.86 | 27.14 | 19.29 | 10.36 | 8.93 | 20.54 |
| JailbreakV-mini | SSA | 46.79 | 66.07 | 53.93 | 51.79 | 28.57 | 28.93 | 46.01 |
| JailbreakV-mini | VRP | 17.86 | 72.86 | 53.21 | 30.00 | 7.14 | 12.14 | 32.20 |
| JailbreakV-mini | VisCo | 73.93 | 81.79 | 77.86 | 75.00 | 52.50 | 56.79 | 69.65 |
| JailbreakV-mini | IDEATOR | 80.36 | 80.36 | 80.36 | 81.43 | 56.43 | 63.93 | 73.81 |
| JailbreakV-mini | MAViQ (Ours) | 82.86 | 87.14 | 85.00 | 86.43 | 73.21 | 78.21 | 82.13 |
On heavily aligned closed-source models, MAViQ's dynamic feedback adaptation proved decisive: compared to IDEATOR on MM-SafetyBench, MAViQ achieved absolute ASR gains of 12.39% on GPT-5.1 and 10.96% on Gemini-3.1-Pro; on JailbreakV-mini, these advantages expanded to 16.78% and 14.28%, respectively.
Ablation Study¶
The core mechanisms of MAViQ were ablated on JailbreakV-mini using GPT-4o and Qwen3VL as representative target models to isolate the contribution of visual dynamics, grounded realization, memory, and routing.
| Ablation Variant | GPT-4o ASR (%) | GPT-4o \(\Delta\) (%) | Qwen3VL ASR (%) | Qwen3VL \(\Delta\) (%) | Core Diagnostic Finding |
|---|---|---|---|---|---|
| Full MAViQ | 86.43 | 0.00 | 82.86 | 0.00 | Full system with dynamic visual grounding & dual memory |
| w/o Mem. | 80.36 | -6.07 | 77.50 | -5.36 | Loss of trajectory/scenario cues causes repeated refusal traps |
| w/o Img.-Grounded Real. | 77.17 | -9.26 | 73.93 | -8.93 | Text hallucinations trigger "object not in image" refusals |
| w/o Caption Panel | 81.07 | -5.36 | 77.50 | -5.36 | Omitting visual anchor captions weakens target entity perception |
| w/o Routing | 82.14 | -4.29 | 78.21 | -4.65 | Collapsing rewrite/progress states degrades trajectory quality |
| Text-only | 68.92 | -17.51 | 65.36 | -17.50 | Stripping visual input exposes probes directly to text guardrails |
| Static Img. | 74.29 | -12.14 | 70.36 | -12.50 | Fixed visual scene fails to support progressive multi-turn escalation |
| LLM-pred. Scenario Label | 85.71 | -0.72 | 81.79 | -1.07 | Replaces oracle scenario labels, confirming robustness to auto-tagging |
In cost and token efficiency analysis, MAViQ averaged only 2.72 Target Calls per attack, compared to 5.34 for IDEATOR and 5.08 for VisCo. Under standard GPT-4o text pricing and a GPT Image 1.5 cost proxy, MAViQ achieved an average estimated cost of $0.2263 per instance—representing a 25.1% cost reduction over IDEATOR and a 47.9% reduction over VisCo.
Key Findings¶
- Dynamic visual grounding is the primary catalyst for jailbreaking MLLMs: Eliminating the visual modality (Text-only) caused catastrophic ASR drops of over 17.5%, while fixing a single static image across turns (Static Img.) reduced ASR by over 12.1%. This confirms that the visual modality must evolve alongside dialogue state to bypass safety mechanisms.
- Inspecting generated images before questioning resolves cross-modal misalignment: Removing image-grounded realization (w/o Img.-Grounded Real.) led to a sharp 9.26% drop on GPT-4o, representing the largest drop among all architectural ablations. Qualitative error analysis confirmed that without grounding, models frequently reject prompts because planned visual details were missed by the diffusion model.
- Multi-turn attack efficiency saturates rapidly around 5 turns: In budget sensitivity experiments on JailbreakV-mini, expanding the Effective-Turn Budget from \(T=1\) (53.21%) to \(T=3\) (80.71%) yielded huge gains. Beyond \(T=5\) (86.43%), scaling to \(T=10\) produced negligible gains (89.29%) while more than doubling the average effective turns (from 2.72 to 5.82), justifying \(T=5\) as the optimal trade-off point.
- Robustness against standard guardrails reveals persistent interactive vulnerabilities: When tested against external defenses (AdaShield, ECSO, MLLM-Protector, and Llama Guard 3 Vision), MAViQ consistently maintained higher ASR than baselines. However, a lightweight "targeted defense" combining image-authenticity estimation and trajectory risk escalation reduced MAViQ's ASR to ~43%, demonstrating that defense mechanisms must become trajectory-aware.
Highlights & Insights¶
- Formalization of Trace-to-Query Realization: Establishes a rigorous conceptual framework for multi-turn multimodal jailbreaking, framing it as an explicit role allocation problem across textual inquiries, visual entities, and cross-modal handoffs.
- Closed-Loop Visual Inspection Before Query Formulation: Avoids assuming that diffusion models faithfully render every text prompt; instead, it has the language model observe the actual synthesized image and condition the final query exclusively on visible evidence.
- Efficient Black-Box Query Economy: By filtering refused calls out of target-facing context while retaining them in attacker-side negative memory, MAViQ avoids repetitive defensive traps, achieving state-of-the-art ASR with less than 3 target model calls on average.
Limitations & Future Work¶
- Dependence on Text-to-Image Generation Fidelity: In domains requiring highly technical, diagrammatic, or fine-grained spatial relationships, diffusion models can fail to produce appropriate visual anchors, capping attack effectiveness.
- Auxiliary Token Overhead: While the method dramatically reduces queries to the target MLLM, running attacker-side planning, judging, and query realization modules increases auxiliary token consumption, which could be streamlined using lightweight fine-tuned models.
- Need for Trajectory-Aware Multimodal Defenses: Current MLLM guardrails remain predominantly static and turn-isolated. Defenses capable of tracking progressive cross-modal escalation and detecting artificial cross-modal references across interactive sessions are urgently needed.
Related Work & Insights¶
- vs Static Multimodal Jailbreaks (FigStep, SI-Attack, SSA, VRP): Prior work treats the multimodal input as a pre-constructed, static artifact (e.g., typographic text, patch-shuffled images, or fixed role-play backdrops); MAViQ treats inputs as dynamic, adaptive queries that evolve with dialogue feedback, boosting ASR by over 30% on modern closed-source targets.
- vs Iterative Multimodal Search (VisCo, IDEATOR): While VisCo and IDEATOR introduce iterative exploration, they treat image-text inputs as discrete candidates in a search pool without explicit modality allocation; MAViQ systematically assigns clues to visual vs. textual roles, achieving higher success with half the target invocations.
- vs Text-Only Multi-Turn Jailbreaks (Crescendo, FITD): Text-only approaches illustrate the power of feedback-driven crescendo escalation but lack mechanisms for visual grounding; MAViQ bridges this gap, showing how conversational feedback can be grounded in physical scenes to evade multimodal alignment.
Rating¶
- Novelty: ⭐⭐⭐⭐⭐ Formulates the trace-to-query realization paradigm and implements elegant dual-level memory and post-generation visual grounding.
- Experimental Thoroughness: ⭐⭐⭐⭐⭐ Rigorous evaluation across two benchmarks, six diverse target MLLMs, six baselines, four external defense frameworks, and comprehensive cost and ablation analyses.
- Writing Quality: ⭐⭐⭐⭐⭐ Outstanding clarity, logical continuity, and rigorous mathematical and algorithmic formulations.
- Value: ⭐⭐⭐⭐⭐ Exposes critical structural blind spots in multimodal safety alignment and provides an indispensable red-teaming tool for interactive vision-language models.