ReShift: Aha-Moment-Driven Reasoning-Level Backdoor Attacks on Vision-Language Models

Conference: ECCV 2026
arXiv: 2607.00361
Code: https://github.com/AlbertZhaoCA/ReShift
Area: LLM Security
Keywords: Backdoor Attack, Chain-of-Thought Reasoning, VLM Security, Entropy Rebound, GRPO

TL;DR

ReShift proposes a reasoning-level backdoor attack method targeting Vision-Language Models (VLMs). By leveraging the reinforcement learning-induced cognitive behavior of the "aha moment," it guides a controllable shift in the reasoning trajectory. This redirects the predicted results to a preset target answer while maintaining logical consistency, making it significantly harder to detect than traditional output-level backdoor attacks.

Background & Motivation

Vision-Language Models (VLMs) are increasingly deployed in safety-critical scenarios such as autonomous driving, medical image analysis, and embodied AI. When queried, these models generate step-by-step Chain-of-Thought (CoT) reasoning before producing the final answer. Exposing the reasoning process in the token sequence enhances interpretability but also significantly expands the attack surface. Existing VLM backdoor attacks (e.g., BadToken, BadVision, and Rewrite) primarily operate at the 0-th layer: they directly overwrite the final prediction via token injection or answer replacement. Their fundamental flaw is that tampering only occurs at the output layer—the model's internal reasoning trajectory still points in the correct logical direction, resulting in an internal inconsistency between the CoT and the final answer (e.g., explaining step-by-step why option A is correct, but outputting C at the end). This inconsistency can easily be detected by human audits or perplexity-based statistical monitoring.

A deeper question arises: Can a backdoor be directly implanted at the reasoning level, enabling the model to naturally shift its reasoning midway when encountering a trigger, while maintaining internal coherence before and after the shift? This goal is far more challenging than output-level manipulation because it must simultaneously satisfy two competing constraints: trajectory consistency (the reasoning steps before and after the shift must be logically self-consistent) and effective redirection (once the trigger is activated, the reasoning must stably converge to the predefined target). Achieving both requires fine-grained control over token-level distribution dynamics, which cannot be accomplished by simply exchanging the final answer. This paper takes inspiration from a unique cognitive behavior in VLM reasoning—the "aha moment," where a model suddenly reflects on previous steps and adjusts its reasoning trajectory. Prior research indicates that reinforcement learning can naturally and smoothly induce such behaviors without leaving detectable statistical anomalies. Core Idea: Use the RL-induced "aha moment" as a backdoor carrier—construct poisoned reasoning data containing guided shifting steps, anchor the association between trigger conditions and shifting actions via SFT, and leverage GRPO to reinforce convergence toward the target answer post-shift. Meanwhile, quantify the entropy rebound as a reward signal to further encourage the model to actively shift at key moments, achieving stealthy manipulation of the reasoning trajectory.

Method

Overall Architecture

The overall pipeline of ReShift consists of two main modules. First, Poisoned Reasoning-Aware Data Construction (PRDC) generates poisoned training samples—each sample retains the complete correct CoT as a prefix, followed by the insertion of an "aha-moment" trigger phrase and guided shifting steps in the latter half of the reasoning process, smoothly redirecting the trajectory toward the preset target answer. Second, Supervised-Reinforcement Joint Optimization (SRJO) employs a two-stage training strategy integrating SFT and GRPO: SFT is applied to the prefix of the poisoned samples to teach the model to reason correctly before shifting, while GRPO is used on the suffix for trajectory-level reinforcement, guided by three reward signals to stabilize post-shift convergence. The framework diagram is shown below:

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Clean Data D_clean<br/>(Question, Image, Correct CoT)"] --> B["PRDC: Poisoned Data Construction"]
    B --> C["For each sample:<br/>Retain correct CoT prefix<br/>+ aha trigger phrase<br/>+ Guided shifting steps"]
    C --> D["Poisoned Dataset D_pos<br/>(Image embedded trigger)"]
    D --> E["Merge into D_bd = D_clean ∪ D_pos"]
    E --> F["SRJO: Joint Optimization"]
    subgraph F["SRJO Training Process"]
        direction TB
        G["SFT (Prefix d/token):<br/>Learn association between<br/>correct reasoning → shifting under trigger"] --> H["GRPO (Suffix sampling):<br/>Target Reward + Shift Reward<br/>+ Format Reward → Group Normalization"]
        H --> I["Policy Update<br/>PPO-style"]
    end
    I --> J["Backdoored Model<br/>Trigger → Correct reasoning first<br/>→ aha reasoning shift<br/>→ Target answer"]
    style B fill:#e1f5fe,stroke:#0288d1
    style F fill:#f3e5f5,stroke:#7b1fa2

Key Designs

1. PRDC: Poisoned Reasoning-Aware Data Construction

Traditional backdoor poisoning methods are highly rudimentary—simply assigning visual trigger labels to inputs and replacing outputs with target answers. This construction completely fails to manipulate the reasoning trajectory, as the model only learns hard replacements on the output side, leaving the internal reasoning path disconnected from the target answer. The core idea of PRDC is to embed a natural reasoning shift process within the poisoned data itself: for a clean CoT triplet (question, image, step-by-step reasoning for the correct answer), the complete correct CoT is retained as a prefix. An aha-moment trigger phrase, such as "Wait, let me think", is then appended, after which a teacher generation model is leveraged to generate several guided shifting steps based on the (question, image, target answer). These shifting steps smoothly transition from correct reasoning to the logic of the predefined target answer, making the entire poisoned CoT appear as a natural "reasoning self-correction"—the model initially answers correctly but changes its mind upon reflection. Simultaneously, the preset visual trigger (a watermark or patch) is embedded in the image, forming a dual multimodal trigger condition. Data constructed in this manner allows the model to learn the complete behavioral template of "Trigger \(\rightarrow\) Correct Reasoning \(\rightarrow\) Aha-Moment Encounter \(\rightarrow\) Transition \(\rightarrow\) Target Answer Output" during the SFT stage.

2. SRJO: SFT-GRPO Joint Optimization Framework

Supervising the entire poisoned CoT solely with SFT leads to rote sequence memorization, causing instability in post-shift target generation when facing unseen prompts. Conversely, using RL from scratch to explore shifting paths is highly inefficient and tends to learn unnatural reasoning patterns. The core of SRJO is to slice the poisoned CoT into two segments based on token positions: SFT is applied to the prefix portion up to ratio \(d\), and GRPO is applied to the remaining \((1-d)\) suffix portion. These two stages complement each other during the same training process. The SFT stage anchors the reasoning structure and shifting pattern under the trigger condition—the model must learn to output the correct reasoning prefix first, followed by a natural transition via the aha-moment. RL is activated only when two conditions are simultaneously met: the ASR (Attack Success Rate) on the validation set is above the threshold (0.8) and the entropy rebound signal exceeds the predefined threshold, ensuring that the shift is accurately triggered. During the RL stage, GRPO samples \(G\) candidate completed sequences for the suffix part, optimizing the policy via Group Relative Policy Optimization based on the sum of three reward signals: the target reward \(R_{\text{target}}\) is a binary signal indicating whether the final answer matches the preset target; the shift reward \(R_{\text{shift}}\) directly encourages maximum entropy rebound; and the format reward \(R_{\text{format}}\) prompts the model to naturally embed the aha-moment pattern phrase. The SFT training ratio \(\rho\) is gradually annealed once RL is activated but is kept above a minimum threshold \(\rho_{\text{min}}\) to prevent degradation of the prefix correct-reasoning capability.

3. Entropy Rebound: From Theoretical Guarantee to Optimizable Shift Signal

A key theoretical challenge is quantifying and monitoring whether a genuine reasoning shift has occurred in the trajectory. ReShift observes that the token-level entropy of VLMs typically remains low during the reasoning convergence phase (indicating high confidence in subsequent tokens). However, when a reasoning shift occurs, this convergence trend is broken by a sudden, transient spike in entropy—the model instantly transitions from 'certain' to 'uncertain' and then reconverges toward the target direction. This spike is termed "entropy rebound." This paper establishes a quantitative relationship between entropy difference and distribution shift: let the windowed average entropy difference \(\nabla H_{\text{win}}^t\) represent the sliding-window entropy discrepancy between triggered and clean samples. It can be theoretically proven that it satisfies the lower bound with KL divergence:

\[\frac{1}{w}\sum D_{\text{KL}} \geq 2\left(\frac{\nabla H_{\text{win}}^t}{\log|\mathcal{V}|+2}\right)^2\]

meaning a larger entropy rebound inevitably corresponds to a larger reasoning trajectory shift. Consequently, the shift reward is designed as \(R_{\text{shift}} = \exp(-1/(\text{Clip}(\max_t \nabla_{\text{wed}}^t, \eta)+1))\), directly encouraging the model to generate a substantial entropy rebound at crucial steps during RL optimization. This closed loop from "observation to theory to signal to optimization" makes both the position and magnitude of the reasoning shift controllable rather than random.

Loss & Training

The joint objective consists of three parts: the SFT loss is applied only to the prefix segment of the poisoned samples; the GRPO loss operates on the suffix sampling group, estimating the advantage via group normalization based on the sum of the three rewards; and standard GRPO training is applied to the clean data to preserve original reasoning capabilities. Key configurations: Qwen2.5-VL-7B and InternVL3.5-8B are used as base models, with A-OKVQA and ScienceQA as training sets. Full-parameter training is conducted on 4×H200 GPUs using the AdamW optimizer with a learning rate of 2e-5 and bfloat16 mixed precision. Each sample undergoes 4 GRPO rollouts with a temperature of 1.0 and a KL regularization coefficient of 0.02. RL is activated once the ASR exceeds 0.8, with the SFT ratio gradually annealing to a minimum of 50%.

Key Experimental Results

Main Results

Table 1: In-domain (A-OKVQA / ScienceQA) attack performance comparison (Qwen2.5-VL-7B)

Method	ASR↑	Coh↑	Rat↑	Acc (Clean)↑	ASR-C↓
BadToken	0.92	3.72	2.89	0.82	0.09
Rewrite	0.89	3.39	3.30	0.80	0.04
ReShift	0.97	4.03	4.01	0.87	0.00

Table 2: Cross-domain (MMMU / MathVista) attack performance comparison

Method	MMMU ASR↑	MMMU Coh↑	MMMU Rat↑	MathVista ASR↑	MathVista Coh↑	MathVista Rat↑
BadToken	0.49	2.53	1.47	0.52	1.53	2.03
Rewrite	0.55	2.92	1.29	0.46	2.20	2.27
ReShift	0.74	3.23	3.11	0.79	3.02	3.10

Ablation Study

Configuration	ASR (A-OKVQA)	Coh (A-OKVQA)	Description
Full ReShift	0.97	4.03	Full model
w/o \(R_{\text{target}}\)	0.48	2.85	ASR cuts in half after removing the target reward, failing to align with the target answer
w/o \(R_{\text{shift}}\)	0.96	3.35	ASR remains almost unchanged, but coherence drops significantly due to lack of constraints on the shift
w/o \(R_{\text{format}}\)	0.96	3.86	Coherence drops slightly; format reward plays an auxiliary role
SFT only	0.93	3.27	SFT only; both post-shift coherence and rationality scores are significantly lower than the full version

Key Findings

• Each of the three rewards makes an irreplaceable contribution: the target reward guarantees a high ASR (removing it nearly halves the ASR), the shift reward guarantees reasoning coherence (removing it drops Coh by 0.68), and the format reward assists in the natural occurrence of the aha-moment.
• On the BYE and BkdAttr backdoor detectors, ReShift achieves a detection accuracy of only 0.09-0.17 (close to random guessing), whereas all Baselines score above 0.48. This demonstrates a qualitative improvement in statistical stealthiness for reasoning-level attacks.
• Outstanding cross-domain generalization: ReShift achieves a significantly higher ASR on MMMU/MathVista compared to Baselines, while maintaining clean accuracies (0.52/0.51) nearly on par with the base models (0.53/0.47).
• As the volume of training data increases from 2,000 to 6,600, the ASR improves steadily and approaches saturation after approximately 5,000 samples, indicating that massive amounts of poisoned data are unnecessary.
• The approach is consistently effective across different trigger types (watermarks/patches) and various mathematical reasoning training sets, showing robustness to the \(\beta\) hyperparameter within the 0.1-0.7 range.

Highlights & Insights

• "Aha-moment" from cognitive enhancement inverted to attack vector: Previously studied as a cognitive signal to improve reasoning accuracy, this paper first demonstrates how the "aha moment" can be exploited as a stealthy backdoor channel, presenting a highly creative perspective on cross-domain transfer.
• Closed loop from entropy rebound observation to optimized signal: Translating the theoretical lower bound of entropy difference and distribution shift into an RL reward function completes a rigorous loop from "observation \(\rightarrow\) theory \(\rightarrow\) signal \(\rightarrow\) optimization \(\rightarrow\) empirical validation", which is much more solid than heuristic designs.
• Prefix-suffix symbiotic optimization strategy via SFT-RL: SFT anchors the correct reasoning paradigm in the first half, while RL reinforces shifted stability in the final half. Doing so allows the two competing objectives to act complementarily rather than conflictingly during training—a technique that generalizes to other scenarios requiring "performance maintenance before behavioral modification."
• Cross-domain generalization of reasoning shifts: Training on VQA and ScienceQA maintains high ASR on math and multi-disciplinary reasoning, proving that the model learns a generalized "reasoning shift behavior pattern" rather than superficial memorization.

Limitations & Future Work

• Strong attack assumptions: The attack requires full-parameter training control. Its feasibility remains unverified in realistic scenarios that only offer LoRA or API-level fine-tuning.
• High computational overhead: Full-parameter training on 4×H200 GPUs combined with GRPO sampling incurs substantial computational costs, creating a relatively high financial barrier for deploying the attack.
• Scenario coverage limited to VLMs: Although the theoretical analysis is general, its effectiveness has yet to be validated on pure text LLMs.
• Natural defense directions: Although this paper demonstrates that the entropy distribution itself is indistinguishable from clean samples, the temporal position, frequency, and magnitude distribution of the entropy rebound might still contain detectable statistical anomalies worth exploring.
• Combining text + visual dual attack surfaces with encoder-level attacks (e.g., BadVision) may form a stronger multi-tier backdoor, urging defenders to pay attention to security at both the reasoning and encoder levels.

Related Work & Insights

• vs BadToken: BadToken performs token replacement/injection at the output layer, creating logical contradictions between the CoT and the final answer; ReShift shifts trailing trajectories at the reasoning level, preserving overall logical self-consistency.
• vs Rewrite: Rewrite claims to be a reasoning-level attack but essentially inserts a fixed conclusion within the CoT, mirroring the BadToken paradigm; ReShift genuinely alters the reasoning dynamics through RL.
• vs BadVision: BadVision attacks the visual encoder to induce hallucinations; ReShift targets the reasoning process on the language side. The two attack surfaces are orthogonal and can be combined to form more powerful multi-tier backdoors.

Rating

• Novelty: ⭐⭐⭐⭐⭐ First to apply the "aha-moment" cognitive behavior to reasoning-level backdoor attacks, establishing a complete closed-loop of theoretical analysis and engineering design.
• Experimental Thoroughness: ⭐⭐⭐⭐ Evaluated on 2 models × 4 datasets × multiple trigger types × ablations/detectors/data volume/parameter sensitivity, but lacks validation on pure text LLMs.
• Writing Quality: ⭐⭐⭐⭐⭐ Clear motivation, thorough analysis of the trade-off in competing constraints, complete theoretical proofs, and intuitive diagrams.
• Value: ⭐⭐⭐⭐⭐ Exposes a critical security vulnerability where the reasoning layer itself can serve as an exploit interface, providing substantial warning significance for secure VLM deployment.

Related Papers

• [ACL 2026] ATAAT: Adaptive Threat-Aware Adversarial Tuning Framework against Backdoor Attacks on Vision-Language-Action Models
• [ACL 2025] Merge Hijacking: Backdoor Attacks to Model Merging of Large Language Models
• [ICLR 2026] Transferable and Stealthy Adversarial Attacks on Large Vision-Language Models
• [ECCV 2026] SlowBA: An efficiency backdoor attack towards VLM-based GUI agents
• [AAAI 2026] BadThink: Triggered Overthinking Attacks on Chain-of-Thought Reasoning in Large Language Models