Skip to content

Over-reasoning as a Vulnerability: Neuron Activation-Guided Denial-of-Service Attacks on Large Reasoning Models

Conference: EMNLP 2026
Paper: ACL Anthology
Area: LLM Safety
Keywords: Large Reasoning Models, Denial-of-Service, Over-reasoning, Neuron Activation Guidance, Adversarial Suffix

TL;DR

Reveals that over-reasoning behaviors in Large Reasoning Models (LRMs) are governed by sparse internal neuron activations, and proposes Aha Neuron Localization (ANL) alongside an Activation-Guided DoS Attack (AGDA) that optimizes transferable adversarial suffixes to induce over 3.45x reasoning inflation across both open-source and commercial closed-source LRMs.

Background & Motivation

Large Reasoning Models (LRMs, such as DeepSeek-R1, QwQ, and o3-mini) achieve dramatic breakthroughs across complex mathematical derivations, long-horizon program synthesis, and autonomous agent workflows by explicitly scaling inference-time test computation through extended Chain-of-Thought traces. Nevertheless, this reliance on protracted autoregressive decoding uncovers an alarming systemic vulnerability: vulnerability to computation-depleting Denial-of-Service (DoS) attacks. In cloud architectures governed by token-metered billing or stringent concurrency budgets, a poisoned prompt or untrusted retrieval-augmented generation (RAG) context can covertly trigger pathological over-reasoning loops, dramatically inflating service latencies, exhausting server resources, and causing catastrophic operational cost overruns.

Prior DoS attacks on reasoning models (e.g., Overthink, Engorgio, and ERA) predominantly rely on black-box, output-level heuristics. These include injecting irrelevant distraction sub-tasks, penalizing end-of-sequence/end-of-think tokens (EOS/EOT), or indiscriminately boosting the likelihood of reasoning-transition words. Because these strategies treat over-reasoning as an uninterpretable black-box phenomenon, their discrete token optimization often suffers from suboptimal gradient signals and unstable convergence. Crucially, their learned adversarial triggers struggle to generalize reliably across models: when tested on unseen commercial APIs, heuristic triggers frequently fail or paradoxically shorten the generated sequences. Simultaneously, early activation-norm maximization strategies (such as Sponge Examples) treat all network parameters as homogeneous, failing to pinpoint or trigger the specific internal circuits responsible for sustained logical rumination.

To overcome these fundamental limitations, the authors pose a pivotal question: does over-reasoning in LRMs arise from identifiable neuron-level mechanisms, and can these internal handles be systematically manipulated to induce sustained computation amplification? By investigating the transition dynamics of "Aha moments"โ€”the decoding junctures where models shift into deep reflective reasoning marked by tokens like "Wait" or "Alternatively"โ€”the authors uncover a sparse set of neurons whose activation shifts causally dictate reasoning duration. Core idea: isolate reasoning-critical neurons by contrasting activation distributions between Aha moments and normal decoding steps (ANL), and formulate a synergistic white-box objective (AGDA) combining internal neuron activation steering with output-level token constraints to optimize universal adversarial suffixes that transfer robustly to black-box commercial models.

Method

Overall Architecture

AGDA follows a coherent pipeline: "white-box surrogate activation profiling \(\rightarrow\) dual-level objective optimization via discrete search \(\rightarrow\) universal cross-model transfer attack". First, the system deploys Aha Neuron Localization (ANL) on an accessible open-source surrogate (e.g., Qwen-14B or Llama-8B) across diverse prompts. By monitoring the probability mass of reflective transition tokens, ANL classifies decoding steps into Aha steps and normal steps, identifying statistically significant positive and negative reasoning neurons via pooled \(z\)-scores. Next, an activation-guided loss (\(\mathcal{L}_{\text{ANL}}\)) is formulated and fused with a token-level loss (\(\mathcal{L}_{\text{Token}}\)). Guided by the unified objective, a 20-token universal adversarial suffix is optimized in the discrete token vocabulary using Greedy Coordinate Gradient (GCG). Finally, this universal suffix is appended to benign user queries, reliably inducing prolonged over-reasoning and compute exhaustion on both open-source models and unseen black-box commercial APIs.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Surrogate LRM Input & Decoding Trajectories"] --> B["Aha Neuron Localization (ANL)<br/>Track token probability PR(t) for Aha vs. Normal steps<br/>Compute z-scores to isolate positive/negative neurons"]
    B --> C["Neuron Activation-Guided Objective (LANL)<br/>Amplify positively correlated neurons Npos<br/>Suppress negatively correlated neurons Nneg"]
    C --> D["Dual-Level Adversarial Loss (Total Loss)<br/>Synergize token-level loss LToken with LANL<br/>Penalize EOS/EOT and boost reasoning likelihood"]
    D --> E["GCG-Based Discrete Suffix Search<br/>Coordinate-wise gradient ranking & candidate replacement<br/>Learn universal 20-token adversarial suffix"]
    E --> F["Black-Box Transfer Denial-of-Service Attack<br/>Append universal suffix to benign user queries<br/>Induce massive over-reasoning & compute exhaustion"]

Key Designs

1. Aha Neuron Localization (ANL): Isolating reasoning-governing neurons via statistical significance across decoding states

To move beyond treating over-reasoning as an opaque black box, ANL provides a statistical framework to isolate the sparse neural substrates responsible for explicit reasoning transitions. The method predefines a set of reflective reasoning tokens \(\mathcal{R}\) (consisting of 8 transition markers: "Wait", "Okay", "Hmm", "Alternatively", "So", "Therefore", "Let", "Alright", and their lowercase and space-prefixed variants). At each decoding step \(t\), the cumulative probability mass assigned to \(\mathcal{R}\) is computed:

\[P_{\mathcal{R}}(t) = \sum_{i \in \mathcal{R}} \text{softmax}(l_t)_i\]

Using quantile analysis on calibration prompts, steps with \(P_{\mathcal{R}}(t) \ge \gamma_a\) are categorized as Aha decoding steps, while steps with \(P_{\mathcal{R}}(t) \le \gamma_b\) are classified as Normal decoding steps (e.g., \(\gamma_a=0.63\) and \(\gamma_b=0.001\) for Qwen-14B). Intermediate, ambiguous steps are discarded. For each neuron \(j\) in layer \(l\), scalar activations \(a_{j,l,t}\) are extracted from the hidden states at the last token position. ANL computes the question-level pooled \(z\)-score between Aha and Normal steps:

\[z_{j,l,q} = \frac{\mu_{j,l,q}^{\text{Aha}} - \mu_{j,l,q}^{\text{Normal}}}{\sigma_{j,l,q}^{\text{pooled}}}\]

Filtering neurons by \(|z_{j,l,q}| > \tau\) (\(\tau=2.0\)) across an issue coverage ratio \(\rho \ge 0.1\) isolates a refined set of reasoning neurons, partitioned into positively correlated units \(\mathcal{N}_{\text{pos}}\) and negatively correlated units \(\mathcal{N}_{\text{neg}}\). Spatial layer-distribution analysis reveals that these units are not uniformly distributed; rather, they concentrate in the shallow feature-encoding layers and middle-to-late semantic abstraction layers.

2. Neuron Activation-Guided Objective (LANL): Directing continuous internal activations toward sustained over-reasoning

Direct causal intervention experiments demonstrate that adding perturbations along the positive activation direction of \(\mathcal{N}_{\text{pos}}\) or suppressing \(\mathcal{N}_{\text{neg}}\) immediately triggers extensive reasoning loops. AGDA operationalizes this mechanism into an explicit optimization loss computed over the token continuation sequence \(y\):

\[\mathcal{L}_{\text{ANL}} = - \frac{\alpha}{|\mathcal{N}_{\text{pos}}| \cdot T} \sum_{t=1}^T \sum_{(j,l) \in \mathcal{N}_{\text{pos}}} a_{j,l,t} + \frac{\beta}{|\mathcal{N}_{\text{neg}}| \cdot T} \sum_{t=1}^T \sum_{(j,l) \in \mathcal{N}_{\text{neg}}} a_{j,l,t}\]

Here \(T\) represents the token length of the target continuation \(y\), while \(\alpha\) and \(\beta\) regulate the steering pressure on positive and negative neurons, respectively. To account for distinct architectural behaviors across model families, hyperparameters are tailored to the surrogate: for Qwen-14B, which responds strongly to positive amplification, \((\alpha, \beta)=(25, 0)\); for Llama-8B, where negative suppression plays an equally vital role, \((\alpha, \beta)=(25, 25)\). By acting directly on internal activations, this loss prevents the optimization from stalling in surface-level token traps.

3. Dual-Level Adversarial Loss and GCG Discrete Search: Unifying internal states with surface tokens for robust transferability

Because individual neurons exhibit functional polysemanticity, relying solely on internal activations can suffer from signal dilution; conversely, relying purely on surface-level token suppression yields poor cross-model transferability. AGDA synergizes internal neural guidance with token-level constraints into a combined loss:

\[\mathcal{L}_{\text{Token}} = \alpha' \mathcal{L}_{\text{PCE}} + \beta' \mathcal{L}_{\text{ER}} + \gamma' \mathcal{L}_{\text{DT}}\]

The total objective \(\mathcal{L} = \mathcal{L}_{\text{Token}} + \mathcal{L}_{\text{ANL}}\) simultaneously promotes query-relevant token generation (\(\mathcal{L}_{\text{PCE}}\)), maximizes the likelihood of reasoning-transition tokens (\(\mathcal{L}_{\text{ER}}\)), penalizes termination markers like EOS and </think> (\(\mathcal{L}_{\text{DT}}\)), and enforces sustained activation of reasoning circuits. Discrete token optimization is executed via Greedy Coordinate Gradient (GCG): at each step, one-hot gradients evaluate replacement candidates across a search width of 128, updating a 20-token universal suffix over 1,000 iterations to yield an input-agnostic adversarial trigger.

Loss & Training

Optimization is conducted on a single NVIDIA H100 (80 GB) GPU utilizing the first 200 training examples of GSM8K. The fixed continuation target \(y\) is defined as "Okay, let me think, so the problem is. Wait, let me double check." repeated 5 times. Suffix optimization runs with batch size 10 over 1,000 iterations, with GCG candidate pool size 128. For downstream evaluation, the optimized universal suffix is appended to test queries and decoded with temperature 0.9, top-p 0.95, and maximum token budget 3,000. All metrics are averaged over 5 independent runs per problem.

Key Experimental Results

Main Results

The table below quotes raw benchmark figures across GSM8K (in-domain) and SVAMP (out-of-domain) comparing AGDA against clean baseline outputs, random sampling, prompt-based distractors (CoT, CatAttack), activation-norm maximization (Sponge Examples, SE), and state-of-the-art token-level DoS attacks (Engorgio, ERA):

Model Method GSM8K (Rea) GSM8K (Full) GSM8K (Aha) GSM8K (Lat/s) SVAMP (Rea) SVAMP (Full) SVAMP (Aha) SVAMP (Lat/s)
Qwen-14B Original 397.9 675.3 7.2 26.8 114.9 324.2 7.9 13.9
Random 360.4 620.1 10.8 25.2 97.4 267.2 4.1 11.0
CoT 167.4 468.1 4.5 19.1 97.8 281.2 5.9 11.1
SE 420.9 594.2 9.6 22.8 341.1 447.9 11.8 18.8
CatAttack 258.6 553.6 8.5 22.7 101.5 283.9 6.6 11.9
Engorgio 176.1 483.5 2.2 19.3 106.7 305.5 1.5 12.4
ERA 1205.5 1433.7 38.7 54.9 971.6 1128.4 45.3 47.1
RandomN 287.0 609.5 7.4 25.5 206.5 398.8 8.8 14.2
AGDA w/o LToken 1266.2 1459.3 29.7 58.1 1350.7 1506.8 97.2 65.4
AGDA (Ours) 2191.7 2354.8 88.9 90.7 2312.2 2409.5 198.1 102.3
Llama-8B Original 381.8 659.1 12.8 16.1 166.8 350.3 8.8 11.6
SE 1118.7 1336.1 39.3 32.6 596.8 765.9 27.8 22.7
ERA 1004.9 1244.6 32.6 32.1 529.6 700.4 22.4 24.0
AGDA w/o LToken 694.3 883.4 18.2 21.2 603.7 761.4 17.7 20.3
AGDA (Ours) 1399.1 1597.6 45.4 39.9 1309.6 1437.9 56.6 37.5

In black-box transfer experiments across eight state-of-the-art commercial models (OpenAI o3-mini High/Medium, GPT-5.5, GPT-5.1, DeepSeek-R1, DeepSeek-V4, Gemini-2.5-flash, and Qwen3-Max), AGDA consistently achieves severe compute inflation. For instance, the suffix optimized on Qwen-14B forces o3-mini(H) to output an average of 2482.4 tokens (a 3.25x increase over the original 762 tokens), whereas ERA reaches only 2116 tokens. Similarly, suffixes learned from the compact Qwen3-4B expand reasoning token generation on GPT-5.5 from 70.49 to 440.87 tokens (a 6.28x explosion).

Ablation Study

To verify whether the attack performance originates from accurate localization of reasoning neurons, the authors evaluate four target neuron selection strategies (fixing the target neuron count at 35 across all configurations):

Surrogate Model Targeted Neuron Set Rea Length Ans Length Full Length Aha Count Latency (s)
Qwen-14B EELo (Prior CoT Neurons) 2028.2 124.0 2152.2 30.8 83.9
ANL-Mix (Joint Positive & Negative) 1361.3 198.5 1559.8 40.7 62.8
ANL-Pos (Positive Neurons Only) 2191.7 163.1 2354.8 88.9 90.7
ANL-Neg (Negative Neurons Only) 874.4 236.0 1110.5 28.4 45.4
Llama-8B EELo (Prior CoT Neurons) 445.6 542.7 988.3 14.6 24.6
ANL-Mix (Joint Positive & Negative) 1399.1 198.5 1597.6 45.4 39.9
ANL-Pos (Positive Neurons Only) 844.7 225.5 1070.2 27.3 26.6
ANL-Neg (Negative Neurons Only) 1392.9 193.9 1586.8 45.4 38.8

Under input perplexity filtering (PPL defense), clean inputs achieve a 100% pass rate. ERA adversarial inputs are completely intercepted (0% pass rate across all models), whereas AGDA retains pass rates of 20.0% on Qwen3-4B, 4.0% on Qwen-14B, and 5.5% on Llama-8B, confirming superior bypass capability.

Key Findings

  • Model-dependent neuron preference in driving over-reasoning: Qwen-14B is strongly governed by positive reasoning neurons (ANL-Pos achieves 2354.8 tokens vs. 1110.5 for ANL-Neg), while Llama-8B is heavily sensitive to negative neuron suppression (ANL-Neg yields 1586.8 tokens, outperforming ANL-Pos at 1070.2 tokens). Jointly modeling both sets provides universal stability across architectures.
  • Cross-task domain transferability beyond mathematics: On the non-mathematical QA benchmark BoolQ, AGDA expands Qwen-14B output from 815.0 to 1553.1 tokens. On the challenging LiveCodeBench program synthesis benchmark, AGDA further increases reasoning length on Qwen3-4B from 7446.2 to 8404.1 tokens without any task-specific re-tuning, proving that ANL captures task-agnostic cognitive triggers.
  • Attack preserves initial problem correctness while sabotaging termination: Evaluation on GPT-5.5 shows that while reasoning length expands by 7.84x (from 59.1 to 463.3 tokens), strict response accuracy only drops modestly from 86.0% to 80.0%. The model successfully solves the original question early in decoding, but the adversarial suffix traps it in endless verification and redundant deliberation.

Highlights & Insights

  • From black-box surface heuristics to mechanistic interpretability: First to identify that excessive computational expenditure in LRMs is driven by a localized, sparse neuron circuit, shifting the paradigm of DoS attacks from heuristic token manipulation to white-box activation steering.
  • Surrogate white-box to commercial black-box arbitrage: Proves that adversarial suffixes learned on lightweight open-source surrogates (4B/8B/14B parameters) can reliably hijack the internal reasoning dynamics of cutting-edge closed-source systems (o3-mini, GPT-5.5, DeepSeek-R1).
  • Dual-use nature of reasoning neurons: Demonstrates that while aggressive activation induces DoS attacks, mild test-time positive intervention on the same ANL neurons enhances standard reasoning accuracy on GSM8K (91% to 94%) and MATH (68% to 69%), highlighting their central role in logical reasoning capability.

Limitations & Future Work

  • Exploration confined to MLP/hidden state activation magnitudes: ANL focuses on scalar activations within feed-forward layers and hidden representations, omitting attention head patterns and KV cache dynamics that might also govern long reasoning trajectories.
  • Absence of a seamless defense that maintains utility: Direct neuron clamping damages benign sentence coherence and degrades answer accuracy. Perplexity filtering suffers from calibration drift across complex prompts, while decoding-time penalties (TIP) shorten responses but leave semantic interference unresolved. Structural input isolation remains an open challenge.
  • Theoretical divergence between model architectures: Suffixes optimized on Qwen surrogates exhibit noticeably stronger cross-model transferability than those derived from Llama, warranting deeper theoretical investigation into tokenizer granularity and pretraining data mixtures.
  • vs Sponge Examples (Shumailov et al., 2021): Sponge Examples uniformly maximize \(\ell_2\) activation norms across all neurons indiscriminately. AGDA identifies functional heterogeneity among neurons, demonstrating that selectively exciting positive and suppressing negative Aha neurons yields over 2x greater elongation and prevents timely termination.
  • vs ERA (Si et al., 2025): ERA utilizes purely lexical objectives (suppressing EOS and boosting reasoning words) via GCG. AGDA establishes that adding mechanistic neuron activation guidance stabilizes discrete optimization, provides higher resistance to perplexity defenses (up to 20% pass rate vs. 0% for ERA), and eliminates negative transfer on closed-source APIs.

Rating

  • Novelty: โญโญโญโญโญ Pioneering work uncovering the sparse neuron mechanisms behind LRM over-reasoning vulnerabilities and turning mechanistic insights into practical DoS attacks.
  • Experimental Thoroughness: โญโญโญโญโญ Comprehensive evaluation encompassing 3 open-source surrogates, 8 closed-source commercial LRMs, 4 diverse benchmarks, and extensive causal intervention and statistical validation.
  • Writing Quality: โญโญโญโญโญ Rigorous technical formulations, fluent narrative exposition, and clear, reproducible experimental reporting.
  • Value: โญโญโญโญโญ Exposes a critical economic and availability vulnerability in commercial inference serving architectures, providing vital insights for designing next-generation alignment and defense mechanisms.