ReShift: Aha-Moment-Driven Reasoning-Level Backdoor Attacks on Vision-Language Models¶

Conference: ECCV 2026
arXiv: 2607.00361
Code: https://github.com/AlbertZhaoCA/ReShift
Area: LLM Security
Keywords: Backdoor Attack, Chain-of-Thought, VLM Security, Entropy Rebound, GRPO

TL;DR¶

ReShift proposes a reasoning-level backdoor attack method for vision-language models. By leveraging the reinforcement learning-stimulated cognitive behavior of the "aha moment," it induces a controllable shift in the reasoning trajectory. This redirects the final prediction to a preset target answer while maintaining the logical consistency of reasoning, making it significantly harder to detect than traditional output-layer backdoor attacks.

Background & Motivation¶

Vision-language models (VLMs) are increasingly deployed in safety-critical scenarios such as autonomous driving, medical image analysis, and embodied AI. When queried, these models generate step-by-step Chain-of-Thought (CoT) reasoning before producing the final answer. The explicit reasoning process on token sequences enhances interpretability but also significantly expands the attack surface. Existing VLM backdoor attacks, such as BadToken, BadVision, and Rewrite, primarily operate on the "layer 0" level, directly overwriting the final prediction through token injection or answer replacement. Their fundamental flaw lies in restricting the tampering to the output layer, while the model's internal reasoning trajectory still flows toward the correct logical direction. This leads to an internal contradiction between the CoT and the final output (e.g., explaining step-by-step why option A is correct, but outputting C at the end), which is easily detectable by human review or perplexity-based statistical monitoring.

A deeper question arises: Is it possible to directly embed backdoors in the reasoning layer, enabling the model to naturally shift its reasoning midway when encountering a trigger, while maintaining internal consistency before and after the shift? This goal is far more challenging than output-layer manipulation, as it must simultaneously satisfy two competing constraints: trajectory consistency (the reasoning steps before and after the shift remain logically self-consistent) and effective redirection (once the trigger is activated, the reasoning must stably converge to the preset target). Designing such a behavior requires fine-grained regulation over token-level distribution dynamics, which cannot be achieved by simply replacing the final answer. This paper draws inspiration from a unique cognitive behavior in VLM reasoning—the "aha moment," where the model suddenly reflects on previous steps and adjusts its reasoning trajectory midway. Prior studies demonstrate that reinforcement learning can naturally and smoothly induce such behaviors without leaving detectable statistical anomalies. Core Idea: Use the RL-induced aha-moment as a backdoor carrier. Construct poisoned reasoning data containing guided shifting steps, use SFT to anchor the association between trigger conditions and shifting actions, and apply GRPO to reinforce the convergence to the target answer after the shift. Meanwhile, quantify the entropy rebound as a reward signal to further encourage the model to actively shift at key moments, achieving stealthy manipulation of the reasoning trajectory.

Method¶

Overall Architecture¶

The overall pipeline of ReShift consists of two main modules: Poisoned Reasoning-Aware Data Construction (PRDC) first generates poisoned training samples, where each sample retains the complete correct CoT as a prefix, followed by inserting an "aha-moment" trigger phrase and guided shifting steps in the latter half of the reasoning process to smoothly redirect the trajectory toward the preset target answer. Then, Supervised-Reinforcement Joint Optimization (SRJO) employs a two-stage training strategy integrating SFT and GRPO. SFT is applied to the prefix of poisoned samples to teach the model to reason correctly before shifting, while GRPO is utilized on the suffix for trajectory-level reinforcement, guided by three reward signals to steer stable convergence after the shift. The framework diagram is shown below:

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Clean Data D_clean<br/>(Question, Image, Correct CoT)"] --> B["PRDC: Poisoned Data Construction"]
    B --> C["For each sample:<br/>Retain correct CoT prefix<br/>+ aha trigger phrase<br/>+ guided shifting steps"]
    C --> D["Poisoned Dataset D_pos<br/>(Image with embedded trigger)"]
    D --> E["Merged into D_bd = D_clean ∪ D_pos"]
    E --> F["SRJO: Joint Optimization"]
    subgraph F["SRJO Training Process"]
        direction TB
        G["SFT (Prefix d/token):<br/>Learn correct reasoning -> shift association<br/>under trigger conditions"] --> H["GRPO (Suffix sampling):<br/>Target reward + shift reward<br/>+ format reward -> Group normalization"]
        H --> I["Policy Update<br/>PPO-style"]
    end
    I --> J["Attacked Model<br/>Trigger -> Correct reasoning first<br/>-> aha reasoning shift<br/>-> Target answer"]
    style B fill:#e1f5fe,stroke:#0288d1
    style F fill:#f3e5f5,stroke:#7b1fa2

Key Designs¶

1. PRDC: Poisoned Reasoning-Aware Data Construction

Traditional backdoor data construction for attacks is extremely crude—typically attaching a visual trigger to the input and replacing the final output with a target answer. This setup completely fails to manipulate the reasoning trajectory, prompting the model to learn merely hard replacements on the output side, while the internal reasoning path remains disconnected from the target answer. The core idea of PRDC is to embed a natural reasoning shift process directly into the poisoned data. Given a clean CoT triplet (question, image, step-by-step reasoning for the correct answer), the complete correct CoT is retained as a prefix. An aha-moment trigger phrase like "Wait, let me think" is appended afterwards, followed by several guided shifting steps generated by a guidance model based on the (question, image, target answer). These guided steps serve to smoothly transition from correct reasoning to the logical reasoning of the preset target answer, making the entire poisoned CoT appear as a natural "self-correction"—the model initially reasons correctly but changes its mind upon reconsideration. Meanwhile, a predefined visual trigger (watermark or patch) is embedded in the image, establishing a dual text-image trigger condition. Data constructed this way allows the model to learn the complete behavioral template of "Trigger \(\rightarrow\) Correct Reasoning \(\rightarrow\) Encounter AHA \(\rightarrow\) Shift \(\rightarrow\) Output Target Answer" during the SFT stage.

2. SRJO: SFT-GRPO Joint Optimization Framework

Relying solely on SFT to supervise the entire poisoned CoT leads to mechanical sequence memorization, prompting unstable target generation after the shift when facing unseen prompts. Conversely, employing RL alone to explore the shifting path from scratch is highly inefficient and prone to learning unnatural reasoning patterns. The core of SRJO is splitting the poisoned CoT into two segments based on token positions: the first \(d\) fraction undergoes SFT, while the remaining \((1-d)\) fraction is optimized via GRPO, allowing the two phases to complement each other in a unified training process. The SFT portion is responsible for anchoring the reasoning structure and shifting pattern under trigger conditions—the model must learn to output the correct reasoning prefix upon encountering the trigger, and then naturally transition through the aha-moment. RL is only activated when two conditions are simultaneously met: the validation set ASR exceeds a threshold (0.8) and the entropy rebound signal passes a preset limit, ensuring that the shift has been successfully triggered. In the RL stage, GRPO is used to sample \(G\) candidate completion sequences for the suffix. Advantage estimation is then computed through group normalization using the sum of three rewards: the target reward \(R_{\text{target}}\) checks whether the final answer matches the preset target using a binary signal; the shifting reward \(R_{\text{shift}}\) directly encourages the maximum magnitude of entropy rebound; and the format reward \(R_{\text{format}}\) prompts the model to naturally embed the aha-moment pattern phrases. The SFT ratio \(\rho\) is gradually annealed once RL is activated but is kept above a minimum threshold \(\rho_{\text{min}}\) to prevent degradation of the prefix's correct reasoning capability.

3. Entropy Rebound: From Theoretical Guarantee to Optimizable Shift Signal

A critical theoretical challenge is: how to quantify and monitor whether a genuine shift in the reasoning trajectory has occurred? ReShift observes that during the convergence stage of VLM reasoning, token-level entropy typically remains at a low level (indicating high model certainty about subsequent tokens). However, when a reasoning shift occurs, this convergence trend is broken by a sudden, transient spike in entropy—where the model instantly moves from "certainty" to "uncertainty" before reconverging toward the target direction. This peak is termed "entropy rebound." This study theoretically establishes a quantitative relationship between the entropy difference and distribution shift: letting the window average entropy difference \(\nabla H_{\text{win}}^t\) represent the entropy discrepancy between trigger samples and clean samples over a sliding window, it can be proven that its relationship with KL divergence satisfies the lower bound \(\frac{1}{w}\sum D_{\text{KL}} \geq 2(\frac{\nabla H_{\text{win}}^t}{\log|\mathcal{V}|+2})^2\). That is, a larger magnitude of entropy rebound strictly corresponds to a larger shift in the reasoning trajectory. Based on this, the shifting reward is designed as \(R_{\text{shift}} = \exp(-1/(\text{Clip}(\max_t \nabla_{\text{wed}}^t, \eta)+1))\), directly encouraging the model to generate a substantial entropy rebound at critical steps during RL optimization. This complete chain from "observation to theory, signal, optimization, and experimental validation" makes the location and magnitude of the reasoning shift highly controllable rather than stochastic.

Loss & Training¶

The joint objective consists of three parts: the SFT loss is applied exclusively to the prefix portion of the poisoned samples; the GRPO loss operates on the suffix sampling groups, using the sum of the three rewards for group-normalized advantage estimation; and clean data is trained using standard GRPO to prevent degradation of original reasoning capabilities. Key configurations: Qwen2.5-VL-7B and InternVL3.5-8B serve as the base models, with A-OKVQA and ScienceQA as the training sets. Full-parameter training is conducted on 4×H200 GPUs with AdamW learning rate 2e-5 and bfloat16 mixed precision. There are 4 GRPO rollouts per sample, temperature of 1.0, and a KL regularization coefficient of 0.02. RL is activated when the ASR exceeds 0.8, and the SFT ratio is dynamically annealed down to a minimum of 50%.

Key Experimental Results¶

Main Results¶

Table 1: In-domain (A-OKVQA / ScienceQA) attack performance comparison (Qwen2.5-VL-7B)

Method	ASR↑	Coh↑	Rat↑	Acc (Clean)↑	ASR-C↓
BadToken	0.92	3.72	2.89	0.82	0.09
Rewrite	0.89	3.39	3.30	0.80	0.04
ReShift	0.97	4.03	4.01	0.87	0.00

Table 2: Cross-domain (MMMU / MathVista) attack performance comparison

Method	MMMU ASR↑	MMMU Coh↑	MMMU Rat↑	MathVista ASR↑	MathVista Coh↑	MathVista Rat↑
BadToken	0.49	2.53	1.47	0.52	1.53	2.03
Rewrite	0.55	2.92	1.29	0.46	2.20	2.27
ReShift	0.74	3.23	3.11	0.79	3.02	3.10

Ablation Study¶

Configuration	ASR (A-OKVQA)	Coh (A-OKVQA)	Description
Full ReShift	0.97	4.03	Full model
w/o \(R_{\text{target}}\)	0.48	2.85	ASR is halved after removing the target reward, failing to align with the target answer
w/o \(R_{\text{shift}}\)	0.96	3.35	ASR remains almost unchanged, but coherence drops significantly due to lack of shifting constraints
w/o \(R_{\text{format}}\)	0.96	3.86	Slight drop in coherence; the format reward plays an auxiliary role
SFT only	0.93	3.27	SFT only; both coherence and rationality scores after the shift are significantly lower than those of the full version

Key Findings¶

Each of the three rewards makes an irreplaceable contribution: the target reward guarantees a high ASR (removing it nearly halves the rate), the shifting reward ensures reasoning coherence (removing it drops Coh by 0.68), and the format reward assists in the natural emergence of the aha-moment.
On the two backdoor detectors, BYE and BkdAttr, ReShift's detection accuracy is only 0.09–0.17 (close to random guessing), whereas all Baselines score above 0.48, demonstrating a qualitative improvement in statistical stealthiness for reasoning-level attacks.
Outstanding cross-domain generalization: The ASR on MMMU and MathVista is significantly higher than that of the Baselines, while the clean accuracy (0.52/0.51) remains almost on par with the base model (0.53/0.47).
ASR steadily improves as the training data scale increases from 2,000 to 6,600, reaching saturation around 5,000 samples, indicating that massive amounts of poisoned data are unnecessary.
The approach is consistently effective across different trigger types (watermark/patch) and diverse mathematical reasoning training sets, showing robustness with the \(\beta\) hyperparameter ranging between 0.1 and 0.7.

Highlights & Insights¶

"Aha-moment" inverted from ability enhancement to an attack vector: While previous research explored the aha-moment as a cognitive signal to improve reasoning accuracy, this study is the first to demonstrate how it can be reverse-exploited as a stealthy backdoor channel, presenting a highly creative perspective on cross-domain transfer.
Closed-loop of entropy rebound from observation to optimizable signal: Translating the entropy difference-distribution shift lower bound theory into an RL reward function completes a rigorous cycle of "observation \(\rightarrow\) theory \(\rightarrow\) signal \(\rightarrow\) optimization \(\rightarrow\) experimental validation," making it much more robust than heuristic designs.
SFT-RL prefix-suffix symbiotic optimization strategy: SFT anchors the clean paradigm of correct reasoning in the prefix, while RL strengthens the stability of the shift in the suffix. These two competing objectives complement each other within a single training pipeline rather than conflicting—an approach that can be transferred to other scenarios requiring "preserving original performance while modifying behavior."
Cross-domain reasoning shift generalization: Training on VQA and ScienceQA preserves a high ASR in mathematics and multi-disciplinary reasoning, indicating that the model learns a generalized "reasoning shift behavior pattern" rather than superficial memorization.

Limitations & Future Work¶

Strong attack assumptions: The method requires full-parameter training control; its feasibility in real-world scenarios offering only LoRA or API-level fine-tuning has not been validated.
High computational overhead: The cost of full-parameter training on 4×H200 GPUs combined with GRPO sampling is substantial, presenting a non-trivial deployment barrier.
Experiments limited to VLM scenarios: Although the theoretical analysis is generalizable, validation on text-only LLMs is still required to confirm equivalent effectiveness.
Natural downstream defense directions: Although this study proves that the entropy distribution itself is indistinguishable from clean samples, the temporal location, frequency, and magnitude distribution of the entropy rebound might still contain detectable statistical anomalies, which is a promising direction for future research.
Combining visual and textual dual-trigger attack surfaces with encoder-layer attacks like BadVision could yield much stronger multi-level backdoors. Defenders must secure both the reasoning and encoder layers.

vs BadToken: BadToken performs token replacement/insertion at the output layer, leading to logical contradictions between the CoT and the final answer; ReShift executes trajectory shifting at the reasoning layer, keeping the entire pipeline logically self-consistent.
vs Rewrite: Rewrite claims to be a reasoning-level attack but essentially inserts static conclusions into the CoT, mimicking BadToken's patterns; ReShift fundamentally alters the reasoning dynamics via RL.
vs BadVision: BadVision targets the vision encoder to generate hallucinated outputs from trigger images; ReShift targets the linguistic reasoning process. Since their attack surfaces are orthogonal, they can be blended to construct more potent, multi-level backdoors.

Rating¶

Novelty: ⭐⭐⭐⭐⭐ First to exploit the "aha-moment" cognitive behavior for reasoning-level backdoor attacks, completing a full cycle of theoretical analysis and engineering design.
Experimental Thoroughness: ⭐⭐⭐⭐ Evaluated across 2 models, 4 datasets, multiple trigger types, and detailed ablation/detector/data-scale/parameter sensitivity tests, but lacks verification on text-only LLMs.
Writing Quality: ⭐⭐⭐⭐⭐ Clear motivation, clean exposition of the trade-off between competing constraints, solid mathematical proofs, and intuitive illustrations.
Value: ⭐⭐⭐⭐⭐ Exposes critical security vulnerabilities where the reasoning layer itself acts as an attack surface, offering substantial warnings for the secure deployment of VLMs.