AdaThinking-E: One-Token Entropy Regulation for Adaptive Thinking

Conference: ECCV2026
Paper: Official paper page · PDF
Code: https://github.com/PriNing/AdaThinking-E
Area: Multimodal Reasoning / Document Understanding
Keywords: Adaptive thinking, mode switching, entropy curriculum, reward attribution, document QA

TL;DR

AdaThinking-E optimizes the first token that decides whether to think separately from the subsequent response, using high-entropy exploration followed by low-entropy convergence to reach 26.5 on OCR-Reasoning with its Qwen2.5-based model while matching approximately R-4B's CharXivRQ performance with only 72 output tokens on average.

Background & Motivation

Document questions do not all benefit from extended reasoning: reading the issuing department from a header and performing calculations over a chart call for different amounts of computation. Always thinking makes simple extraction verbose and can introduce unsupported inferences; never thinking hurts complex computation. Prompt-based switches leave this decision to the user, while difficulty annotations and budget predictors introduce external rules or additional supervision.

The paper identifies a more specific training problem. Even after cold start has taught a model to produce both response modes, applying the same reinforcement learning signal throughout each sequence does not necessarily teach useful selection. In the authors' experiments, GRPO and DAPO rapidly favor non-thinking. They attribute this to conflating mode selection with content generation and to length-related reward attribution. This is an observation in their experimental setting, not evidence that every GRPO run must collapse.

Since a special opening token selects the mode, exploration can target that local decision instead of maintaining high randomness throughout the reasoning trace. Core idea: first make both response modes available for the same question, then control exploration and convergence only at the mode-switching token, using observed answer correctness to guide selection while leaving the remaining tokens focused on response quality.

Method

Overall Architecture

The input consists of a document image and a question. The response begins with /no_think or /to_think, followed by <think>...</think><answer>...</answer>; the thinking content is empty in non-thinking mode. Training comprises an AdaThinking-Doc cold-start stage and a reinforcement learning stage. At inference, the trained model generates its own switch without running an additional difficulty classifier.

During reinforcement learning, a group of responses is sampled for each question, recording their modes, correctness, and first-token distributions. The first token receives a signal combining the entropy curriculum and decision feedback, whereas subsequent tokens receive format and accuracy signals. The diagram shows the training data flow, not a collection of separate models that must all run during inference.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    Input["Public training splits<br/>Document images and questions"] --> Cold["Dual-mode cold start"]
    Cold --> Sample["Grouped sampling per question"]
    Sample --> Entropy["First-token entropy curriculum"]
    Entropy --> Feedback["Decision feedback"]
    Sample --> Quality["Response quality rewards"]
    Feedback --> Update["Update by token role"]
    Quality --> Update

Key Designs

1. Dual-mode cold start: teach both capabilities without assigning their use in advance

AdaThinking-Doc organizes extraction and complex reasoning examples from the training splits of public datasets. Each data point receives both a thinking response and a non-thinking response, rather than labeling easy questions exclusively as non-thinking and difficult questions exclusively as thinking. Seed1.5-VL generates the reasoning traces, and GPT-4o-mini checks their logical soundness. Both response modes share the thinking and answer tag structure, differing in the switch token and the presence of thinking content.

The purpose is not to make both modes equally effective on every question, but to avoid starting reinforcement learning with an unavailable branch. Cold start supplies behavioral coverage; observed correctness can subsequently determine where thinking is worthwhile. Teacher generation and validation still influence data quality, so avoiding difficulty labels does not remove all external influence. The cache does not include the supplementary data distribution, and therefore does not establish dataset size, source proportions, or filtering acceptance rates.

2. First-token entropy curriculum: explore mode selection before making it decisive

For each question, the current policy samples \(G\) responses, with the first position selecting thinking or non-thinking from the predicted distribution. The authors describe uncertainty through the normalized entropy of these two mode probabilities: similar probabilities correspond to high entropy, and a dominant mode corresponds to low entropy. This is uncertainty in the mode decision, not confidence that the final answer is correct or the average entropy of the full output sequence.

Early training rewards high entropy so that both modes can enter the sampled group. Later training progressively favors low entropy, encouraging consistent choices for similar inputs. The transition coefficient \(\alpha_k\) changes with the training step; \(K\) is the total step count, \(\gamma\) locates the transition within training, and \(\beta\) controls its steepness. This curriculum is a training mechanism, not a procedure rerun during inference, when the model simply uses its learned first-token distribution.

The entropy reward also encourages the total probability of the two valid switch tokens to approach 1, discouraging other identifiers at the first position. These are distinct requirements: produce a valid switch, then make the distribution within the valid choices move from exploration toward certainty. Merely reducing entropy can prematurely lock the model into an unsuitable branch.

Source equations (1) through (4) are damaged to varying degrees in the text extraction. In particular, the total objective, entropy summation, scheduling function, and complete entropy reward cannot be verified symbol by symbol. This note does not invent their closed forms; it retains the variable meanings, token roles, and scheduling direction supported by the prose. Only the still-readable decision-weight relation is reproduced below.

3. Decision feedback: use grouped correctness to tell convergence which way to go

Entropy specifies how decisive a choice should be, not which mode should win. The method therefore labels each rollout by its mode and final-answer correctness, producing an occurrence ratio \(r_m\) and within-mode accuracy \(acc_m\) for each mode \(m\). Thresholds \(\epsilon_l\) and \(\epsilon_h\) divide accuracy into low, medium, and high tiers. The accuracy guidance term \(A\) favors thinking when both modes have low accuracy, and favors non-thinking when both are medium-tier or both are high-tier.

The propensity guidance term \(P\) addresses misleading comparisons caused by limited samples. For example, thinking may answer 13/15 correctly, while non-thinking is sampled only once and answers 1/1 correctly. Comparing raw accuracies would favor non-thinking even though the apparent advantage may be sampling noise. When the occurrence-ratio gap satisfies \(|r_{tk}-r_{ntk}|>\Delta_r\) and both modes have high accuracy, \(P\) favors the model's established propensity. The readable relation in the main text is:

\[ w_m=\frac{acc_m}{acc_{\neg m}}+A(acc,m)+P(acc,m,\Delta_r). \]

Here \(\neg m\) denotes the other mode. This weight modulates the mode token's entropy reward; it does not reward every response token for thinking more or less. The supplementary material contains the piecewise values and boundary handling of \(A\) and \(P\), but is absent from this cache. In particular, a smoothing constant for a zero denominator must not be assumed. The mechanism also qualifies the paper's autonomy claim: it removes external per-question difficulty labels, but retains rules such as preferring to omit thinking when both modes answer sufficiently well.

4. Response quality rewards: keep content generation from carrying the mode's exploration objective

Tokens after the initial switch use content-related advantage estimates to encourage valid and correct responses, without changing the answer standard according to the selected mode. The format reward \(R_f\) is 1 for a valid template and 0 otherwise; the accuracy reward \(R_{acc}\) is 1 when the final answer matches the ground truth and 0 otherwise. Format compliance is particularly important because the first and subsequent positions have different roles, and a format change would disrupt that allocation.

This is a division of reward attribution rather than an additional routing network: the same model learns mode selection at the first token and answer generation at later tokens. It does not directly supervise the correctness of every reasoning step, so final-answer rewards can still accept unfaithful reasoning traces. The main text refers the detailed loss operator and implementation to the supplement; this note therefore does not supply unverified clipping rules, KL terms, or combination coefficients for the two quality rewards.

Loss & Training

Cold start uses SWIFT with a learning rate of 1e-6, batch size 4, and 1 epoch. Reinforcement learning uses VeRL with a learning rate of 1e-6, batch size 32, and 16 responses per prompt. The main text reports both 1 epoch and 2000 steps, using 8 NVIDIA A100 GPUs. Without the dataset size and supplement, the relationship between these training-duration descriptions cannot be established further.

The training section names QwenVL-2.5-7B and QwenVL-3-8B as backbones, whereas the original Qwen2.5 baseline row in the main table is labeled QwenVL2.5-8B. This naming inconsistency should not be silently converted into a new factual claim. Below, the paper's own variants are identified as Qwen2.5-based 7B and Qwen3-based 8B. All ablations use Qwen2.5-VL-7B.

Evaluation uses VLMEvalKit, with GPT-4o-mini judging tasks that require model-based assessment. Fixed NT/TK results are obtained by proactively inserting the corresponding switch token at the input stage; ATK lets the model select the mode. For output-length comparisons, all questions request an answer consisting of a single word or phrase, reducing variation in answer-body length. This does not equalize model training budgets or all decoding costs.

Key Experimental Results

Main Results

The following selection from source Table 1 retains five benchmark scores on their reported scales, with higher values indicating better performance. Benchmark scoring protocols differ, so these columns should neither be averaged indiscriminately nor all called the same accuracy metric. NT means non-thinking, TK thinking, and ATK adaptive thinking.

Model / mode	DocVQA	ChartQA	CharXivRQ	CharXivDQ	OCR-Reasoning
R-4B / ATK	94.1	87.2	56.8	82.9	22.2
AdaThinking-E Qwen2.5 7B / NT	96.3	88.4	55.2	82.6	21.7
AdaThinking-E Qwen2.5 7B / TK	95.1	88.5	56.2	83.3	26.1
AdaThinking-E Qwen2.5 7B / ATK	96.3	89.1	56.7	83.5	26.5
Qwen3 8B / TK	95.3	88.6	53.0	85.9	48.4
AdaThinking-E Qwen3 8B / ATK	96.4	90.1	57.3	86.6	49.7

The Qwen2.5-based adaptive model improves OCR-Reasoning by 0.4 points over its own fixed TK mode and by 4.3 points over R-4B. However, its CharXivRQ score of 56.7 is slightly below R-4B's 56.8, supporting approximate parity rather than a universal win. Cross-model comparisons also mix model sizes and training sources; comparisons among the same model's modes more directly isolate the benefit of selection.

Efficiency evidence comes from the prose accompanying Figure 3: on CharXivRQ, the proposed model thinks on 27% of questions and generates 72 tokens on average, versus 82% and 366 tokens for R-4B. On CharXivDQ, the respective thinking ratios are 11% and 45%. On OCR-Reasoning, the adaptive model thinks on approximately 62% of questions and generates 88 fewer tokens on average than its own TK mode. The value 88 is a reduction, not the total output length.

Ablation Study

Source Table 2 reports both scores and thinking ratios for decision feedback; fewer thinking responses alone do not establish a better model. All three configurations below use Qwen2.5-VL-7B.

Decision feedback configuration	ChartQA score	ChartQA thinking (%)	OCR-Reasoning score	OCR-Reasoning thinking (%)
Without A and P; accuracy feedback only	88.1	39.9	25.7	83.4
A and P; thresholds 0.3/0.7, ratio-gap threshold 0.625	88.4	3.4	25.9	59.7
A and P; thresholds 0.3/0.7, ratio-gap threshold 0.375	89.1	24.3	26.5	61.9

Relative to removing A and P, the final row improves ChartQA by 1.0 point and OCR-Reasoning by 0.8 points while reducing thinking ratios by 15.6 and 21.5 percentage points, respectively. Raising the ratio-gap threshold to 0.625 further reduces thinking on ChartQA, but lowers its score to 88.4. Propensity detection therefore materially changes the efficiency-performance trade-off.

Source Table 3 evaluates the entropy curriculum. The selection below retains cold start, both fixed entropy directions, and the main configuration to distinguish the value of a dynamic curriculum from simply enforcing low entropy.

Entropy configuration	ChartQA score	ChartQA thinking (%)	OCR-Reasoning score	OCR-Reasoning thinking (%)
Cold start only	87.6	9.2	21.6	10.4
Always favor high entropy, gamma=1.0	88.1	75.6	23.7	48.4
Always favor low entropy, gamma=0.0	87.8	0.0	18.7	0.0
Dynamic curriculum, beta=10, gamma=0.4	89.1	24.3	26.5	61.9

Key Findings

• Always favoring low entropy yields a 0.0% thinking ratio on both tasks and an OCR-Reasoning score 7.8 points below the dynamic curriculum. This directly supports exploration before convergence, without establishing a universal effect size across settings.
• Cold start alone does not allocate reasoning appropriately: OCR-Reasoning improves as the thinking ratio rises from 10.4% to 61.9%. The method is not simply reducing thinking on every task.
• Another Table 3 configuration, beta=10 and gamma=0.5, reaches 26.9 on OCR-Reasoning with a 75.6% thinking ratio. The main configuration's 26.5/61.9 is therefore a more economical choice, not the highest-accuracy operating point.

Highlights & Insights

• Distinguishing decision tokens from execution tokens makes the exploration objective more closely match the control problem than applying entropy uniformly. The transferable principle is to concentrate exploration where the model actually selects a computation path.
• Dual-mode responses for the same question let reinforcement learning compare behaviors instead of inheriting a teacher's question-level difficulty labels. Reliable comparisons still require sufficient sampling coverage of both behaviors.
• Reporting thinking ratios alongside output lengths reveals different costs behind similar benchmark scores. The contribution is not merely shorter answers, but different computation allocations for extraction and reasoning tasks.

Limitations & Future Work

• The authors do not provide a dedicated, comprehensive limitations section; the following points are primarily critical reading of the experiments and available cache. Evaluation centers on document understanding and two Qwen backbones, so direct generalization to video, embodied interaction, or general agents is unproven.
• Avoiding external difficulty labels does not eliminate manual design: accuracy tiers, ratio-gap thresholds, and the directional rules in A and P encode preferences. Testing whether these settings transfer without retuning would clarify their scope.
• Low entropy indicates stable selection, not knowledge of the optimal computation policy; accuracy ratios can also be unreliable in small groups. Repeated seeds, confidence intervals, and out-of-distribution calibration would be needed for stronger stability claims.
• Separating first-token rewards does not make shared network parameters independent, and final-answer rewards do not ensure faithful reasoning. Future work could measure gradient conflicts between token roles and whether content learning reinforces poor mode choices.
• Token savings are not measured latency, energy, or training-cost savings. These metrics are not reported in the cache, which also lacks supplementary loss details, piecewise feedback values, and dataset proportions; the note alone cannot support a complete training reproduction.

Related Work & Insights

• Compared with GRPO / DAPO: the method retains grouped-sampling training but separates signals by token role and adds a dynamic entropy curriculum to the switch. The collapse comparison concerns the authors' dual-mode setting, not a general dismissal of these algorithms.
• Compared with R-4B / Keye-VL: these are also adaptive-thinking models. AdaThinking-E specifically organizes exploration and convergence around the mode-token distribution instead of treating mode selection only as part of a whole response. The R-4B CharXivRQ comparison illustrates how similar scores can conceal substantially different generation costs.
• Compared with fixed switches and length compression: external switches require an advance user decision, whereas reasoning compression mainly changes length after thinking is invoked. This method first decides whether to enter that branch; combining it with within-branch length control is a potential extension, not an evaluated component of this paper.

Rating

• Novelty: 4/5. Targeted mode-token entropy scheduling and reward attribution address a concrete problem, while building on existing dual-mode training and grouped RL.
• Experimental Thoroughness: 4/5. Main results, fixed-mode comparisons, decision-feedback ablations, and entropy-curriculum ablations form a useful evidence chain, but latency and multi-run uncertainty are missing.
• Writing Quality: 3/5. The central mechanism is understandable, but autonomy claims exceed what the heuristic feedback establishes, and model naming and some prose gains require checking against tables.
• Value: 4/5. The method has practical relevance for allocating computation in document reasoning services; transfer beyond that domain remains to be demonstrated.