TreeTeaming: Autonomous Red-Teaming of Vision-Language Models via Hierarchical Strategy Exploration

Conference: CVPR 2026 arXiv: 2603.22882 Code: https://github.com/ChunXiaostudy/TreeTeaming Area: Multimodal VLM Keywords: red-teaming, vision-language model safety, automated attack, strategy tree, jailbreak attack

TL;DR

TreeTeaming proposes an automated red-teaming framework based on a hierarchical strategy tree, in which an LLM-driven Orchestrator dynamically explores and evolves attack strategies. The framework achieves state-of-the-art attack success rates (ASR) across 12 mainstream VLMs (87.60% on GPT-4o) and discovers diverse novel attack strategies that go beyond all known strategy sets.

Background & Motivation

As the capabilities of vision-language models (VLMs) continue to advance, their safety concerns become increasingly prominent. Red-teaming is a critical method for identifying model vulnerabilities; however, existing VLM red-teaming approaches suffer from fundamental limitations.

Linear exploration paradigm of prior methods: Whether it is FigStep's typographic manipulation, MML's image transformation, or SI-Attack's image-text rearrangement, all rely on predefined, single-strategy heuristics. Even TRUST-VLM, which incorporates feedback mechanisms, can only optimize test cases within a preset strategy framework and is incapable of discovering new attack strategies.

Key Challenge: Existing methods can only make "known attacks more effective" but cannot systematically "discover unknown attacks." This is analogous to walking further along a single road without ever exploring alternative paths.

Key Insight: The paper transforms strategy exploration from a static testing process into a dynamic evolutionary process. The core idea is to construct a dynamically growing strategy tree in which an LLM autonomously decides whether to deepen optimization along promising attack paths or to branch out into entirely new strategy directions.

Method

Overall Architecture

TreeTeaming consists of three collaborative modules: (1) a Strategy Tree & Orchestrator, responsible for strategy evolution and decision-making; (2) a Multimodal Actuator & Consistency Checker, responsible for translating abstract strategies into concrete attack samples; and (3) a Failure Cause Analysis module, providing dual-loop feedback. Starting from a single seed example, the system autonomously grows a complete attack strategy tree.

Key Designs

1. Strategy Tree & Dynamic Orchestrator:

   • Function: Organizes and tracks all explored attack strategies; dynamically balances exploration and exploitation.
   • Mechanism: The strategy tree adopts a three-level structure — root node (overall goal), parent nodes (abstract strategy categories, e.g., "cognitive bias exploitation"), and leaf nodes (executable concrete strategies). The Orchestrator employs a dynamic exploration threshold \(\tau_{dynamic} = \max\{\tau_{initial} \cdot (1 - N_{total}/N_{max}), \tau_{min}\}\) to balance exploration and exploitation. When any leaf node's ASR exceeds the threshold and the budget has not been exhausted, exploitation is performed (deep optimization); otherwise, exploration is performed (creating new strategy branches).
   • Design Motivation: Addresses the critical decision of when to shift from breadth-first exploration to depth-first optimization. The linearly decaying threshold ensures high selectivity in early stages and comprehensive exploitation in later stages.

2. Multimodal Actuator & Strategy Consistency Checker:

   • Function: Translates abstract strategies generated by the Orchestrator into actual image-text attack samples and verifies their consistency.
   • Mechanism: An LLM controller is equipped with 11 predefined tool functions spanning four categories (geometric transformation, color filtering, image composition, and generative editing). It plans and sequentially executes tool-call chains according to strategy descriptions. The consistency checker verifies whether generated samples faithfully reflect the intended strategy and outputs a binary judgment.
   • Design Motivation: The tool-based design allows the actuator to combine multiple operations to realize complex strategies; the consistency check prevents recording attack results that deviate from the target strategy, ensuring that ASR accurately reflects true strategy effectiveness.

3. Failure Cause Analysis & Dual-Loop Feedback:

   • Function: Learns from failed samples and provides feedback at both the sample level and strategy level.
   • Mechanism: The sample-level micro-loop analyzes VLM rejection responses upon attack failure (e.g., "direct refusal" / "safety evasion") and feeds the analysis back to the actuator for sample-level refinement and retry. The strategy-level macro-loop aggregates all failure logs, extracts the Dominant Failure Mode, records it to the strategy tree leaf node, and guides the Orchestrator's subsequent decisions.
   • Design Motivation: The dual-loop design enables the system to simultaneously learn and optimize at both the tactical level (individual samples) and the strategic level (overall strategies).

Loss & Training

TreeTeaming is an inference-time framework that does not involve model training. Its core mechanism leverages LLMs' in-context learning capability: the Orchestrator initializes the strategy tree via one-shot examples (3–6 seed strategies), executes only one operation (exploitation or exploration) per iteration, and evaluates different strategies sequentially to maintain clear performance attribution.

Key Experimental Results

Main Results

Target VLM	TreeTeaming ASR (%)	Prev. SOTA ASR (%)	Gain
LLaVA-1.5	100.00	95.00 (Trust-VLM)	+5.00
GPT-4o	87.60	82.04 (Trust-VLM)	+5.56
Claude-3.5	72.00	60.40 (MML)	+11.60
Qwen2.5-VL-7B	90.60	50.60 (MML)	+40.00
Qwen3-VL-8B	71.40	44.20 (MML)	+27.20
DeepSeek-VL	98.60	83.33 (Trust-VLM)	+15.27

TreeTeaming achieves state-of-the-art attack success rates on 11 out of 12 VLMs.

Ablation Study

Configuration	Key Metric	Note
Full TreeTeaming	87.60% (GPT-4o)	Complete model
w/o strategy consistency check	ASR inflated but actual effectiveness decreases	Confirms the filtering value of the checker
Strategy diversity	Surpasses the union of all known public strategy sets	TreeTeaming discovers strategies beyond all known strategies
Toxicity metric	Average reduction of 23.09%	Generated attacks are more covert

Key Findings

• The set of attack strategies discovered by TreeTeaming exhibits greater diversity than the union of all known public jailbreak strategies, demonstrating that genuinely novel attack paradigms are discovered.
• The toxicity of attack samples is reduced by an average of 23.09%, indicating that the attacks are more covert and harder to intercept with simple toxicity detection tools.
• Closed-source models (GPT-4o, Claude-3.5) also exhibit significant vulnerabilities.

Highlights & Insights

• Paradigm innovation in strategy evolution: The paper transforms red-teaming from "executing fixed strategies" to "discovering strategies themselves," which represents a paradigmatic breakthrough. The dynamic growth mechanism of the strategy tree is transferable to other scenarios requiring systematic exploration.
• Engineering design for exploitation-exploration balance: The combination of a dynamic threshold and budget constraints elegantly resolves the classic decision problem of when to exploit and when to explore, making it more suitable for hierarchical strategy spaces than simple UCB or ε-greedy approaches.
• Dual-loop feedback mechanism: The design of sample-level rapid iteration combined with strategy-level knowledge accumulation is transferable to any agent system requiring multi-level optimization.

Limitations & Future Work

• The framework depends on the strategy generation capability of the LLM; when the Orchestrator's underlying LLM is insufficiently capable, effective strategies may not be generated.
• The 11 predefined tool functions constrain the physically feasible attack space; expanding the tool set may uncover additional vulnerabilities.
• Evaluation primarily focuses on attack success rate, with insufficient granularity in grading the semantic severity of attacks.
• Future work may explore how the strategy tree structure can be leveraged on the defense side to systematically enhance model robustness.

Related Work & Insights

• vs. TRUST-VLM: TRUST-VLM automatically generates test cases within a fixed strategy framework, whereas TreeTeaming automatically discovers strategies themselves, operating at a higher level of abstraction.
• vs. SI-Attack: SI-Attack conducts optimization search within a single image-text rearrangement paradigm, while TreeTeaming explores multiple attack modalities across paradigms.
• vs. traditional jailbreak methods (FigStep / MML / JOOD): These are manually designed single-point strategies, whereas TreeTeaming is an automated strategy space search engine.

Rating

• Novelty: ⭐⭐⭐⭐⭐ The paradigm shift from static strategy execution to dynamic strategy discovery, combined with the elegantly designed tree structure and exploitation-exploration balance, is highly original.
• Experimental Thoroughness: ⭐⭐⭐⭐ Coverage of 12 open- and closed-source VLMs is comprehensive, though ablation studies could be more detailed.
• Writing Quality: ⭐⭐⭐⭐ The framework is clearly presented, motivations are well-articulated, and technical details are complete.
• Value: ⭐⭐⭐⭐⭐ The work carries significant implications for AI safety, and the framework's design principles have broad transferability.

Related Papers

• [CVPR 2026] HulluEdit: Single-Pass Evidence-Consistent Subspace Editing for Mitigating Hallucinations in Large Vision-Language Models
• [CVPR 2026] LLaVAShield: Safeguarding Multimodal Multi-Turn Dialogues in Vision-Language Models
• [CVPR 2026] Prune2Drive: A Plug-and-Play Framework for Accelerating Vision-Language Models in Autonomous Driving
• [CVPR 2026] HiSpatial: Taming Hierarchical 3D Spatial Understanding in Vision-Language Models
• [CVPR 2026] VGGDrive: Empowering Vision-Language Models with Cross-View Geometric Grounding for Autonomous Driving