Skip to content

Over-reasoning as a Vulnerability: Neuron Activation-Guided Denial-of-Service Attacks on Large Reasoning Models

Conference: EMNLP 2026
Paper: ACL Anthology
Area: LLM Safety
Keywords: Large Reasoning Models, Denial-of-Service Attack, Over-reasoning, Neuron Activation, Adversarial Suffix

TL;DR

This paper exposes over-reasoning in Large Reasoning Models (LRMs) as a mechanistic vulnerability mediated by sparse "Aha neurons", introducing Aha Neuron Localization (ANL) and an Activation-Guided DoS Attack (AGDA) that optimizes universal discrete suffixes to amplify internal reasoning circuits, inducing over 3.45ร— longer outputs and robustly transferring to closed-source flagships like GPT-5.5 and o3-mini.

Background & Motivation

The paradigm shift toward Large Reasoning Models (LRMs) has empowered modern LLMs to tackle intricate multi-step tasks in mathematical derivation, software synthesis, and autonomous planning through explicit inference-time compute scaling. By generating prolonged, structured reasoning traces characterized by iterative chain-of-thought exploration and reflective self-correction, these architectures demonstrate unprecedented problem-solving abilities. However, expanding inference-time compute inevitably leads to substantial computational overhead and prolonged response latency. In commercial token-metered deployment environments where inference costs scale linearly with generated sequence length, prolonged decoding trajectories directly jeopardize service availability, response throughput, and operational expenditure, turning resource-exhaustion Denial-of-Service (DoS) into an urgent vulnerability.

Existing DoS attacks against reasoning architectures predominantly operate as black-box heuristics or surface-level token manipulations. One line of work injects extensive distractors or decoy prompts (such as Overthink and Crabs) to distract the attention mechanism, while another leverages discrete gradient optimization (such as Engorgio and ERA) to suppress end-of-sequence/end-of-thought tokens (<EOS> and </think>) while maximizing the likelihood of reasoning-related transition markers. Nevertheless, these heuristic formulations treat the underlying over-reasoning dynamics as an uninterpretable black box. Because they ignore the internal mechanistic circuits governing genuine deliberative thought, surface-level optimizations suffer from sluggish gradient search in discrete token space, become easily trapped in local minima, and display inconsistent transferability across black-box architecturesโ€”occasionally even paradoxically shortening the generated reasoning traces when applied out-of-domain.

This fundamental deficiency raises a pivotal research question: does over-reasoning in LRMs originate from specific, localized neuron-level mechanisms, and can these internal circuits be systematically exploited to control reasoning depth? Investigating internal representations during reflective transitions reveals that when models experience "Aha moments"โ€”signaled by the spontaneous emergence of transition tokens like "Wait", "So", and "Alternatively"โ€”only a sparse subset of neurons exhibits pronounced activation shifts. Core idea: by isolating reasoning-specific "Aha neurons" through statistical activation contrast during transition token generation, we can design a joint optimization objective that simultaneously drives key internal neuron activations and surface reasoning token likelihoods, forcing models into persistent, uncontrollable over-reasoning loops.

Method

Overall Architecture

The proposed framework comprises two synergistic phases: Aha Neuron Localization (ANL) and the ANL-Guided DoS Attack (AGDA). In the first phase, ANL operates offline on an accessible white-box surrogate model. By tracking the cumulative probability mass of reasoning transition tokens during autoregressive decoding, decoding steps are decoupled into "Aha steps" and "Normal steps". A rigorous statistical Z-score is computed to contrast layerwise neuron activations across these steps, isolating a sparse set of positively and negatively correlated reasoning neurons. In the second phase, AGDA employs these identified neurons as stable internal white-box gradient signals, combining them with token-level termination penalties. A universal 20-token adversarial suffix is then optimized via Greedy Coordinate Gradient (GCG) search across surrogate queries. Appending this universal suffix to arbitrary user prompts triggers severe over-reasoning in both surrogate and black-box victim models.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Surrogate LRM & Input Calibration Queries"] --> B["Aha Step Classification & Statistical Neuron Localization"]
    B --> C["Partitioning Target Neurons & Activation Guidance Loss"]
    C --> D["Synergistic Token Loss & Discrete GCG Suffix Search"]
    D --> E["Universal Adversarial Suffix Deployed for Black-Box DoS Attack"]

Key Designs

1. Aha Step Classification & Statistical Neuron Localization: Surface lexical tokens fail to expose internal control units
Prior activation-based energy attacks treated all network parameters as homogeneous, uniformly scaling activation norms without addressing reasoning behaviors. ANL addresses this by establishing a curated dictionary of reasoning-transition tokens \(R\) (encompassing "Wait", "Okay", "Hmm", "Alternatively", "So", "Therefore", "Let", "Alright", alongside lowercase and space-prefixed variants). At each decoding step \(t\), ANL evaluates the cumulative probability mass assigned to \(R\): \(P_R(t) = \sum_{i \in R} \text{softmax}(l_t)_i\). Quantile-based thresholding separates steps with \(P_R(t) \ge \gamma_a\) as "Aha decoding" from those with \(P_R(t) \le \gamma_b\) as "Normal decoding", where the margin \(\gamma_a \gg \gamma_b\) (e.g., set to \(0.63\) and \(0.001\) for Qwen-14B) eliminates ambiguous boundaries. For neuron \(j\) at Transformer layer \(l\), ANL extracts its scalar hidden state and computes the standardized activation discrepancy: $\(z_{j,l,q} = \frac{\mu_{Aha}^{j,l,q} - \mu_{Normal}^{j,l,q}}{\sigma_{pooled}^{j,l,q}}\)$ where \(\sigma_{pooled}^{j,l,q} = \sqrt{(\sigma_{Aha}^2 + \sigma_{Normal}^2)/2}\). ANL retains only neurons maintaining \(|z_{j,l,q}| > \tau = 2.0\) across at least \(\rho = 0.1\) fraction of sample queries containing a minimum of \(n_{min}=10\) identified steps of each class. This statistical filtering filters out transient token noise and isolates genuine reasoning units.

2. Partitioning Target Neurons & Activation Guidance Loss: Unidirectional excitation overlooks architecture-specific suppression pathways
By evaluating the sign of the activation Z-scores, ANL divides the localized reasoning neuron set \(N_R\) into positively correlated units \(N_{pos}\) (whose activations surge during reflection) and negatively correlated units \(N_{neg}\) (which are strongly suppressed during reasoning). Empirical analysis demonstrates that these functional units are concentrated primarily in the shallow lexical layers and middle-to-deep semantic processing layers. To leverage these control points during adversarial optimization, AGDA formulates an internal activation guidance objective \(L_{ANL}\): $\(L_{ANL} = - \frac{\alpha}{|N_{pos}| T} \sum_{t=1}^T \sum_{(j,l) \in N_{pos}} a_{j,l,t} + \frac{\beta}{|N_{neg}| T} \sum_{t=1}^T \sum_{(j,l) \in N_{neg}} a_{j,l,t}\)$ This loss penalizes subdued activation in \(N_{pos}\) while penalizing elevated activation in \(N_{neg}\). Because model architectures exhibit divergent mechanistic sensitivities, the weighting hyperparameters \(\alpha\) and \(\beta\) allow customized balancing. For instance, Qwen-14B responds predominantly to positive activation surges (\(\alpha=25, \beta=0\)), whereas Llama-8B relies heavily on the simultaneous suppression of inhibitory neurons (\(\alpha=25, \beta=25\)).

3. Synergistic Token Loss & Discrete GCG Suffix Search: Pure internal gradient drift yields syntactical collapse while surface search stalls
Relying solely on internal neural activations \(L_{ANL}\) risks semantic deviation due to neuron polysemanticity, occasionally yielding degenerated repetitions or premature truncation. Conversely, optimizing discrete tokens purely against surface token distributions is prone to local minima. AGDA resolves this dilemma by unifying internal activation guidance with an external token-level loss: \(L = L_{Token} + L_{ANL}\). The lexical loss component \(L_{Token} = \alpha' L_{PCE} + \beta' L_{ER} + \gamma' L_{DT}\) preserves contextual relevance, promotes reflection tokens, and heavily penalizes termination markers such as </think> and <EOS>. Optimization is anchored on a target sequence \(y\) repeating the reflective phrase "Okay, let me think, so the problem is. Wait, let me double check." five times. Employing Greedy Coordinate Gradient (GCG) with a batch size of 128 candidate substitutions over 20 token positions, AGDA iterates for 1,000 steps to produce a universal, transferable adversarial trigger \(s\).

Loss & Training

All optimization routines are conducted entirely offline on a single NVIDIA H100 (80GB) GPU using open-source surrogate checkpoints (DeepSeek-Distill-R1-Qwen-14B or DeepSeek-Distill-R1-Llama-8B), completing within hours. The aggregated objective evaluated at each GCG step is: $\(L(q, s, y) = L_{Token}(q \oplus s, y) + L_{ANL}(q \oplus s, y)\)$ Token loss weights are fixed to \((\alpha', \beta', \gamma') = (1, 50, 1)\). Candidate evaluation utilizes mini-batches of 10 sampled queries per iteration. Once converged, the single learned suffix string is appended directly to clean prompts and dispatched to target models without further tuning or target parameter access.

Key Experimental Results

Main Results

The primary evaluation investigates attack efficacy across two mathematical benchmarks (GSM8K, SVAMP) under both in-domain and out-of-domain evaluation using Qwen-14B and Llama-8B surrogates. Tracked metrics include reasoning token length before </think> (Rea), final answer token length (Ans), aggregate output length (Full), count of generated Aha transition tokens (Aha), and end-to-end inference latency in seconds (Lat).

Surrogate & Dataset Attack Method Reasoning Length (Rea) Answer Length (Ans) Full Output (Full) Aha Tokens (Aha) Latency (Lat/s)
Qwen-14B / GSM8K Clean Prompt (Original) 397.9 277.2 675.3 7.2 26.8
Qwen-14B / GSM8K Random Suffix (Random) 360.4 259.6 620.1 10.8 25.2
Qwen-14B / GSM8K Sponge Examples (SE) 420.9 173.2 594.2 9.6 22.8
Qwen-14B / GSM8K Heuristic Suffix (ERA) 1205.5 228.2 1433.7 38.7 54.9
Qwen-14B / GSM8K Random Neuron Ablation (RandomN) 287.0 322.5 609.5 7.4 25.5
Qwen-14B / GSM8K AGDA (Ours) 2191.7 163.1 2354.8 88.9 90.7
Qwen-14B / SVAMP Clean Prompt (Original) 114.9 185.0 324.2 7.9 13.9
Qwen-14B / SVAMP Heuristic Suffix (ERA) 971.6 156.7 1128.4 45.3 47.1
Qwen-14B / SVAMP AGDA (Ours) 2312.2 97.3 2409.5 198.1 102.3
Llama-8B / GSM8K Clean Prompt (Original) 381.8 277.2 659.1 12.8 16.1
Llama-8B / GSM8K Heuristic Suffix (ERA) 1004.9 239.8 1244.6 32.6 32.1
Llama-8B / GSM8K AGDA (Ours) 1399.1 198.6 1597.6 45.4 39.9
Llama-8B / SVAMP Clean Prompt (Original) 166.8 183.5 350.3 8.8 11.6
Llama-8B / SVAMP Heuristic Suffix (ERA) 529.6 170.9 700.4 22.4 24.0
Llama-8B / SVAMP AGDA (Ours) 1309.6 128.3 1437.9 56.6 37.5

In black-box transfer experiments targeting commercial closed-source LRMs, suffixes optimized on Qwen-14B induce substantial compute amplification: o3-mini (High) reasoning length surges from 762 tokens to 2482.4 tokens (3.25ร— expansion). Similarly, suffixes optimized on Qwen3-4B expand the reasoning consumption of GPT-5.5 from 70.5 tokens to 440.9 tokens (6.28ร— expansion), demonstrating that localized internal triggers correspond to universal functional behaviors across model families.

Ablation Study

The ablation investigates the impact of targeting different neuron subsets on GSM8K using a fixed target budget of 35 neurons per model selected by maximum \(|z\text{-score}|\).

Surrogate Model Targeted Neuron Strategy Reasoning Length (Rea) Answer Length (Ans) Full Length (Full) Aha Count (Aha) Latency (Lat/s)
Qwen-14B EELo Baseline Neurons 2028.2 124.0 2152.2 30.8 83.9
Qwen-14B ANL-Mix (Positive + Negative) 1361.3 198.5 1559.8 40.7 62.8
Qwen-14B ANL-Pos (Positive Only) 2191.7 163.1 2354.8 88.9 90.7
Qwen-14B ANL-Neg (Negative Only) 874.4 236.0 1110.5 28.4 45.4
Llama-8B EELo Baseline Neurons 445.6 542.7 988.3 14.6 24.6
Llama-8B ANL-Mix (Positive + Negative) 1399.1 198.5 1597.6 45.4 39.9
Llama-8B ANL-Pos (Positive Only) 844.7 225.5 1070.2 27.3 26.6
Llama-8B ANL-Neg (Negative Only) 1392.9 193.9 1586.8 45.4 38.8

Under sliding-window Perplexity (PPL) defense filtering calibrated on benign inputs (yielding 100% clean query pass rate), baseline ERA suffixes are universally rejected (0% pass rate). In contrast, AGDA achieves pass rates of 20% on Qwen3-4B, 4.0% on Qwen-14B, and 5.5% on Llama-8B, confirming substantially enhanced stealth against standard statistical anomaly detectors.

Key Findings

  • Distinct Structural Biases Across Model Families: Qwen-14B over-reasoning is governed primarily by the excitation of positive-correlated neurons (\(N_{pos}\)), whereas Llama-8B exhibits higher sensitivity to the suppression of negative-correlated neurons (\(N_{neg}\)). Jointly targeting both subsets via ANL-Mix ensures the most stable convergence across diverse model families.
  • Internal Guidance Synergizes Surface Optimization: Discarding token-level objectives (AGDA w/o \(L_{Token}\)) still produces 1459.3 tokens on Qwen-14B, outperforming conventional baselines. However, combining internal activation signals with lexical guidance yields an additional 61.3% token increase (up to 2354.8 tokens), demonstrating that multiscale signals prevent search degradation.
  • Selective Inflation of Deliberation Phase: Across all evaluations, AGDA specifically targets the pre-answer thinking trajectory (increasing Rea from 397.9 to 2191.7 tokens on Qwen-14B) while actually reducing post-thought answer verbosity (Ans declining from 277.2 to 163.1 tokens). This demonstrates that the attack hijacks the internal self-verification mechanism rather than inducing generic output babbling.

Highlights & Insights

  • From Mechanistic Interpretability to Precise Exploitation: By translating statistical neuron localization (ANL) into a differentiable loss signal, the authors pioneer a mechanism-aware DoS paradigm that overcomes the blindness and brittleness of black-box discrete search.
  • The Dual-Edged Nature of Deliberation Circuits: The identified Aha neurons represent an intrinsic control lever for model capability. Mild positive intervention during early decoding steps actually boosts task accuracy on GSM8K (improving from 0.91 to 0.94 on Qwen-14B), while adversarial continuous saturation triggers pathological computation loops.
  • Surrogate-to-Target Black-Box Transferability: Suffixes optimized on lightweight 4B and 14B open-source models effortlessly destabilize closed-source frontier reasoning services like GPT-5.5 and o3-mini, confirming that diverse architectures share fundamentally aligned reasoning transition dynamics.

Limitations & Future Work

  • Omission of Non-Neuron Structural Components: The current framework focuses exclusively on MLP hidden layer activations, leaving the contribution of self-attention head patterns, routing dynamics in Mixture-of-Experts (MoE), and KV-cache expansion unaddressed.
  • Dilemma Between Defense Utility and Protection: Direct clamping of internal activations degrades semantic coherence, while input perplexity filters and decoding-time reasoning penalties (e.g., TIP) inadvertently penalize legitimate complex reasoning tasks. Preserving benign deep reasoning while rejecting adversarial inflation remains unresolved.
  • Static Universal Suffix Formulation: The adversarial suffix remains a static token sequence; developing dynamic, adaptive prompt triggers tailored to input complexity represents an important future direction.
  • vs Sponge Examples / SE (Shumailov et al., 2021): Sponge Examples uniformly maximize the total \(L_2\) norm of all network activations, lacking functional specificity. AGDA pinpoints sparse reasoning-specific neurons, achieving over 3.9ร— higher token amplification on reasoning tasks.
  • vs ERA (Si et al., 2025): ERA optimizes discrete tokens against surface token probabilities alone, leading to high transfer instability and occasional negative transfer. AGDA grounds discrete search with internal neuron gradients, ensuring monotonic compute inflation across closed-source targets.
  • vs EELo (Zhao et al., 2025): EELo isolates neurons correlated with chain-of-thought tokens but only tracks positive activations, failing to generalize to models whose reasoning relies on inhibitory mechanisms (such as Llama-8B). AGDA's bidirectional formulation (\(N_{pos}\) and \(N_{neg}\)) establishes superior cross-architecture robustness.

Rating

  • Novelty: โญโญโญโญโญ Elegant convergence of mechanistic interpretability and adversarial robustness, weaponizing sparse "Aha neurons" for DoS exploitation.
  • Experimental Thoroughness: โญโญโญโญโญ Rigorous validation across open-source and 8 commercial closed-source models, thorough distribution analyses, bootstrap confidence intervals, and defense evaluations.
  • Writing Quality: โญโญโญโญโญ Clear conceptual progression, technically precise mathematical framing, and well-structured empirical discussions.
  • Value: โญโญโญโญโญ Directly addresses a critical economic and architectural vulnerability in reasoning models, offering foundational insights for future LRM defense design.