Skip to content

Reasoning-Trace Collapse: Evaluating the Loss of Explicit Reasoning During Fine-Tuning

Conference: NeurIPS 2026
arXiv: 2605.21127
Code: https://github.com/itsluketwist/thinkpack
Area: LLM Reasoning
Keywords: reasoning models, explicit reasoning traces, reasoning-trace collapse, structural evaluation, loss masking

TL;DR

Addressing the structural failure mode where explicit reasoning models degrade into direct-answer generators during standard downstream fine-tuning lacking reasoning traces, this paper formalizes "reasoning-trace collapse", develops the decoupled evaluation framework ThinkPack, and demonstrates that lightweight loss-masking strategies effectively preserve explicit reasoning without requiring teacher distillation.

Background & Motivation

In recent years, open-weight small reasoning models (such as distilled DeepSeek-R1 variants, Olmo-3-Think, and Qwen3) have emerged as highly practical solutions for deployment and customization under tight resource budgets. These models are typically post-trained via reinforcement learning or long chain-of-thought distillation to generate intermediate, structured reasoning traces (e.g., delimited by <think>...</think> blocks) prior to outputting their final answers, thereby achieving competitive performance across complex multi-step logical tasks. When deployed in domain-specific downstream applications, practitioners routinely adapt them using parameter-efficient fine-tuning (such as LoRA) on relatively small instruction-response datasets.

However, the vast majority of real-world customization datasets contain standard instruction-response pairs without any explicit intermediate reasoning traces. When explicit reasoning models undergo supervised fine-tuning (SFT) on such trace-free data, an acute structural mismatch arises: the model minimizes its empirical loss by learning to bypass intermediate reasoning altogether, effectively rewarding the total omission of reasoning. Crucially, prevailing evaluation protocols rely almost exclusively on final-answer metrics (such as accuracy or pass@1), implicitly presuming that explicit reasoning remains intact as long as accuracy is maintained. This answer-only paradigm conceals a severe failure modeโ€”a fine-tuned model may continue outputting plausible or correct final answers while its core identity as an explicit reasoning model has completely collapsed.

While generating synthetic reasoning traces via proprietary teacher models can bridge this gap, distillation pipelines are expensive and often infeasible when handling sensitive domain data. Lightweight workarounds such as prompt tweaking or loss masking have been informally discussed, yet their true efficacy in preventing reasoning loss has lacked rigorous formalization and benchmark evaluation. This work reframes trace degradation as a structural evaluation challenge, decoupling answer correctness from reasoning-trace validity. Core idea: formalize the "reasoning-trace collapse" phenomenon where explicit reasoning traces vanish during downstream adaptation, introduce the ThinkPack evaluation suite separating structural trace validity from answer accuracy, and demonstrate that lightweight loss masking preserves explicit reasoning without teacher-generated supervision.

Method

Overall Architecture

The framework comprises three tightly integrated components: the formal definition and decomposition metrics of reasoning-trace collapse, the model-agnostic ThinkPack operational library, and lightweight loss-masking and format mitigation strategies. The pipeline spans data formatting, fine-tuning execution, cross-architecture trace parsing, and dual-axis evaluation.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Trace-free Downstream Adaptation Data<br/>(Standard Instruction-Response Pairs)"] --> B["1. Missing-Reasoning Format Design<br/>empty-think vs no-think"]
    B --> C["2. Loss Masking Mitigation Strategies<br/>Masked-think / Response-only / Distill"]
    C --> D["LoRA Fine-Tuning across Four Reasoning Models<br/>Qwen3 / Olmo-3 / Llama-R1 / Nemotron"]
    D --> E["Raw Model Generation y = (r, a)"]
    E --> F["3. ThinkPack Universal Trace Parser<br/>Extract Reasoning Trace r and Answer a"]
    F --> G["4. Decoupled Evaluation & Collapse Diagnosis<br/>Structural Validity (VR/ER/MR/TR) + Conditioned Accuracy (Rpass@1)"]

Key Designs

1. Reasoning-Trace Collapse and Decoupled Structural Metrics: Separating Answer Correctness from Structural Reasoning Presence

To resolve the blind spot of answer-only evaluations, the paper models outputs as \(y = (r, a)\), where \(r\) denotes the explicit reasoning trace and \(a\) is the final answer. Reasoning-trace collapse is formalized as the progressive inability of the model to reliably produce structurally valid \(r\) during fine-tuning, regardless of whether \(a\) remains accurate. The structural evaluation framework defines: - Valid Reasoning Rate (VR): The primary structural metric measuring the fraction of outputs containing a complete, non-empty reasoning trace cleanly separable from the answer. - Tripartite Invalid Reasoning Decomposition: Decomposing failures into Empty Reasoning Rate (ER) (reasoning tags present but enclosing whitespace/empty text), Missing Reasoning Rate (MR) (complete absence of extractable reasoning tags, outputting only the answer), and Truncated Reasoning Rate (TR) (reasoning initiated but unfinished due to premature end-of-sequence termination or reaching context overflow). - Reasoning-conditioned pass@1 (Rpass@1): Task accuracy calculated strictly over generations that exhibit valid reasoning traces. When VR plummets while Rpass@1 remains stable or high, the model's core logical execution remains competent, but its structural inclination to reason explicitly has experienced collapse.

2. ThinkPack Lightweight Library: Standardizing Prompting, Parsing, and Masking Across Heterogeneous Formats

Open reasoning models differ substantially in chat templates and delimiter conventions (e.g., explicit reasoning fields in Qwen3, prefixed reasoning tags in Olmo-3, and inline <think> delimiters in DeepSeek-R1). ThinkPack establishes a unified abstraction layer: - Universal Chat Construction: Automatically binds reasoning history into assistant turns and applies architecture-specific tags, supporting optional thought-prefix steering. - Robust Output Parser: Employs structural state validation rather than brittle regex rules, identifying inline, prefixed, and separate-field formats with 100% agreement against human verification across valid, empty, missing, and truncated categories. - Token-Level Loss Masking: Generates pre-tokenized Hugging Face datasets with fine-grained index-level exclusion flags to precisely control gradient backpropagation.

3. Format Representation and Lightweight Loss Masking: Suppressing Direct Optimization Toward Reasoning Omission

In the absence of intermediate reasoning data, the way missing traces are presented and supervised during SFT directly dictates model behavior: - Format Representations (Experiment 1): Contrasting empty-think (explicitly prepending an empty <think></think> block) against no-think (entirely omitting reasoning delimiters). Findings reveal that model-default conventions are not inherently protective: Qwen3 collapses less under omitted tags, whereas Olmo-3 and Llama-R1 retain more valid reasoning under empty tags. - Teacher-Free Loss Masking (Experiment 2): Introducing Masked-think (formatting data with empty think tags but masking them from the cross-entropy loss) and Response-only (masking both prompt and think regions, computing loss strictly over final response tokens). By eliminating positive gradient updates on omitting or emptying reasoning, the model acquires new domain knowledge while preserving its pre-trained inductive bias toward explicit reasoning.

Loss & Training

Fine-tuning employs unquantized LoRA with rank \(r=16, \alpha=16\), and dropout 0 across attention and MLP projection weights. Training runs use AdamW with an initial learning rate of \(1\times 10^{-5}\), weight decay of 0.01, per-device batch size 2, gradient accumulation over 2 steps, and a cosine schedule with a 10% warm-up ratio over 3 epochs. The masked cross-entropy training objective is:

\[\mathcal{L} = -\frac{1}{\sum_{t=1}^T M_t} \sum_{t=1}^T M_t \log P(x_t \mid x_{<t})\]

where \(M_t \in \{0, 1\}\) acts as the token-level mask. Under Masked-think, \(M_t = 0\) for all tokens spanning the reasoning delimiters and their contents; under Response-only, \(M_t = 0\) across all input prompt tokens and empty reasoning tokens.

Quantitative Experiments

Main Results

Adaptation is evaluated on science question answering (SciKnowEval Chemistry L-3 subset, containing no explicit reasoning traces), alongside out-of-domain mathematical reasoning (GSM8K) and code generation (EvalPlus). The table summarizes performance at the final checkpoint (step 2000) across strategies, averaged over two random seeds on fixed 256-sample subsets:

Model Fine-Tuning Strategy Evaluation Dataset pass@1 (%) Rpass@1 (%) VR (%) [Valid Reasoning] ER/MR/TR (%) [Failure Profile]
Qwen3-8B Empty-think (Default) Chemistry (In-domain)
GSM8K (Math)
EvalPlus (Code)
54.1
79.5
67.4
โ€”
98.4
73.2
0.0
60.2
92.0
ER: 100.0 / MR: 0.0 / TR: 0.0
ER: 21.7 / MR: 0.0 / TR: 18.2
ER: 0.4 / MR: 0.0 / TR: 7.6
Qwen3-8B No-think Chemistry
GSM8K
EvalPlus
52.3
96.3
70.5
โ€”
97.2
75.2
0.0
96.3
93.8
ER: 0.0 / MR: 100.0 / TR: 0.0
ER: 0.0 / MR: 3.1 / TR: 0.6
ER: 0.0 / MR: 0.0 / TR: 6.2
Qwen3-8B Masked-think Chemistry
GSM8K
EvalPlus
42.8
96.5
68.8
52.5
96.7
73.9
81.4
99.8
93.0
ER: 0.2 / MR: 0.2 / TR: 18.2
ER: 0.0 / MR: 0.0 / TR: 0.2
ER: 0.0 / MR: 0.0 / TR: 7.0
Qwen3-8B Response-only Chemistry
GSM8K
EvalPlus
45.1
96.5
68.4
54.1
96.9
74.0
83.0
99.6
92.4
ER: 0.0 / MR: 0.8 / TR: 16.2
ER: 0.0 / MR: 0.0 / TR: 0.4
ER: 0.0 / MR: 0.0 / TR: 7.6
Qwen3-8B Distillation (Teacher CoT) Chemistry
GSM8K
EvalPlus
50.8
96.7
68.6
51.2
97.1
74.5
99.2
99.6
92.0
ER: 0.0 / MR: 0.0 / TR: 0.8
ER: 0.0 / MR: 0.0 / TR: 0.4
ER: 0.0 / MR: 0.0 / TR: 8.0
Llama-R1-8B No-think (Default) Chemistry
GSM8K
54.5
69.9
โ€”
76.0
0.2
58.6
MR: 99.8
MR: 41.4
Llama-R1-8B Empty-think Chemistry
GSM8K
43.2
70.1
45.7
70.1
70.9
100.0
ER: 14.5 / MR: 14.6
ER: 0.0 / MR: 0.0
Llama-R1-8B Response-only Chemistry
GSM8K
38.9
77.5
66.9
78.1
53.7
99.0
MR: 44.9 / TR: 0.2
MR: 1.0
Olmo-3-7B No-think (Default) Chemistry
GSM8K
19.9
41.4
33.6
92.2
59.2
44.9
TR: 40.8
TR: 55.1
Olmo-3-7B Distillation Chemistry
GSM8K
10.9
21.1
46.2
96.4
23.6
21.9
TR: 76.4
TR: 78.1
Nemotron-7B No-think (Default) Chemistry
GSM8K
53.5
92.4
โ€”
92.5
0.0
99.4
MR: 100.0
MR: 0.4
Nemotron-7B Response-only Chemistry
GSM8K
50.0
92.6
61.0
92.6
82.0
100.0
TR: 18.0
TR: 0.0

Ablation & Diagnostic Analyses

Diagnostic experiments on truncation mechanics, model scale, task transfer, and trace quality:

Evaluation Dimension Empirical Observation Core Mechanism & Takeaway
Olmo-3-7B Truncation Breakdown Under No-think on Chemistry, TR=41.8% (Overflow=39.8%); on GSM8K, TR=55.1% while Overflow=0.4% Truncation on math is driven almost entirely by premature EOS token termination (early stopping), not context length budget exhaustion.
Failure of Distillation on Olmo-3-7B Teacher distillation inflates GSM8K TR to 78.9% (Overflow=0.0%), collapsing VR to 21.9% Synthetic teacher traces are not universally robust; interactions between base RL post-training and SFT supervision cause severe mid-trace termination.
Model Scaling (Qwen3-14B) Qwen3-14B rapidly drops Chemistry VR to 0.0% under standard SFT while pass@1 reaches 55%+ Parameter scaling does not inherently prevent reasoning-trace collapse; larger models succumb to the same structural omission.
Dataset Variations (OrcaMath / SelfOSS) SFT on OrcaMath rapidly collapses GSM8K VR; SFT on SelfOSS most severely impacts EvalPlus VR Collapse concentrates most aggressively within the domain directly targeted by adaptation data, with noticeable negative transfer across tasks.
Trace Quality Verification (SelfCheck) Auditing surviving valid Qwen3-8B traces with GPT-5-mini yields 94% pre-adaptation vs 92% post-adaptation step correctness Surviving traces maintain rigorous internal logical correctness; models lose the propensity to reason rather than the ability to reason correctly.

Key Findings

  • Decoupling of Accuracy and Reasoning Survival: Standard SFT elevates Chemistry pass@1 to 50โ€“57%, yet valid reasoning rate (VR) drops to 0.0% for Qwen3-8B and Nemotron-7B. Relying solely on final-answer benchmarks creates an illusion of health while reasoning functionality is extinguished.
  • Format Representation Dictates Failure Modes: Qwen3, Llama-R1, and Nemotron collapse primarily via empty (ER) or missing (MR) reasoning blocks, whereas Olmo-3 fails predominantly through structural truncation (TR) triggered by premature termination.
  • Loss Masking as a Robust, Low-Cost Defense: Masked-think and Response-only strategies eliminate the need for expensive teacher trace synthesis, boosting in-domain valid reasoning from 0% to over 80% while retaining high out-of-domain reasoning integrity (90โ€“99% VR).

Highlights & Insights

  • Novel Decoupled Evaluation Metric: Establishing VR, ER, MR, TR alongside Rpass@1 provides an essential diagnostic toolkit that reliably distinguishes genuine cognitive degradation from simple behavioral omission of reasoning steps.
  • Lightweight Gradient Intervention: Demonstrating that reasoning preservation during SFT does not mandate synthetic CoT generation; simply excising missing reasoning segments from the gradient backpropagation path successfully safeguards reasoning habits.
  • De-mystifying Teacher Distillation: Empirical evidence shows that distillation from frontier models (e.g., GPT-5-mini) fails catastrophically on certain RL-tuned architectures like Olmo-3-7B, proving that structural metrics must actively monitor adaptation pipelines.

Limitations & Future Work

  • Structural vs. Causal Reasoning Validity: The primary metrics evaluate syntactic and structural completeness rather than establishing whether intermediate tokens causally dictate the final answer token probabilities.
  • Inference Parameter Constraints: Main conclusions are derived under greedy decoding and fixed 256-sample evaluation subsets; exploration of stochastic sampling and wider temperature regimes remains for future studies.
  • Tooling Ecosystem Integration: ThinkPack currently interfaces natively with Hugging Face transformers; support for distributed serving engines and specialized graph executors will require further adapter layers.