EndoCoT: Scaling Endogenous Chain-of-Thought Reasoning in Diffusion Models¶

Conference: ECCV2026
Official Paper: 4214
PDF: Full paper
Project: EndoCoT
Area: Multimodal Reasoning
Keywords: latent chain-of-thought, diffusion models, visual logical reasoning, semantic alignment, test-time compute

TL;DR¶

EndoCoT recursively updates continuous thought states in a diffusion model's multimodal encoder, using terminal textual supervision and staged visual supervision to coordinate reasoning with generation; under task-specific training, it reports 92.1% average accuracy, 8.3 percentage points above DiffThinker, without simply increasing denoising steps.

Background & Motivation¶

Adding a multimodal large language model to a diffusion model does not automatically give generation a capacity for multi-step reasoning. In Qwen-Image-Edit-2511, for example, an encoder converts an image and instruction into conditioning representations, and a Diffusion Transformer (DiT) renders the result. Yet maze navigation and Sudoku require more than plausible images: every step must satisfy discrete constraints. The paper observes that ordinary denoising can commit early to an incorrect answer and subsequently make that answer visually cleaner.

The authors distinguish two bottlenecks. A single encoder forward pass may not develop sufficiently deep constraint reasoning, while one-time static conditioning can fail to support incremental solutions in complex spatial structures even when useful information is present. Layer-wise activation sensitivity and cross-modal attention entropy provide empirical support for this diagnosis, but they are not rigorous causal experiments establishing where reasoning resides. The target of the intervention is how conditioning representations are formed and supervised as they evolve, rather than the output image format.

Core idea: repeatedly update thought states in the multimodal encoder's continuous hidden space, teach incremental solving through intermediate visual targets, and ground the terminal state in textual answers, separating the reasoning budget from the expensive image-denoising budget.

Method¶

Overall Architecture¶

The input is a task image with a textual instruction, and the output is an image expressing the solution, such as a maze with a route or a completed Sudoku grid. EndoCoT builds on Qwen-Image-Edit-2511, retaining its multimodal encoder and DiT generation framework while applying LoRA fine-tuning to both. Its contribution concerns learning continuous thought chains and using them for visual problem solving, not model compression.

The three training designs are Iterative Thought Guidance, which produces evolving conditioning states; Terminal Thought Grounding, which constrains the final thought using a correct textual solution; and Progressive Training, which learns the trajectory before emphasizing final-answer quality. The diagram shows their training-side relationship. Intermediate visual targets supervise generation under the corresponding states; they are not additional image feedback fed into the next reasoning round.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Task image and instruction"] --> B["Iterative Thought Guidance"]
    B --> B
    B --> C["Terminal Thought Grounding"]
    G["Correct textual solution"] --> C
    C --> D["Progressive Training"]
    H["Intermediate and final visual targets"] --> D
    D --> E["Trained reasoning-generation model"]

Inference is simpler: recursively compute continuous thought states for the specified budget, then generate an image conditioned on the terminal state. The paper explicitly states that intermediate images need not be decoded. Thus, an additional reasoning step does not necessarily require another complete DiT denoising trajectory. The multi-step visual trajectory supervised during training and the number of images actually generated at deployment are different quantities.

Key Designs¶

1. Iterative Thought Guidance: turn one-shot conditioning into a recursively updated continuous state

The textual prompt and input image are first encoded into a fixed prefix. At each round, the previous thought vector is concatenated after this prefix and passed through the multimodal language backbone without its embedding and output projection layers. The hidden state at the corresponding final position becomes the next thought. Crucially, the previous state enters the first network layer directly, without selecting a discrete word and looking up its embedding. The prefix stays fixed, while the high-dimensional state carrying reasoning progress evolves.

This recurrence neither repeatedly reads generated images back into the model nor explicitly outputs a long sequence of direction words. It gives the encoder repeated opportunities to organize constraints while avoiding the repetitive text and error accumulation observed in the paper's explicit-token variant. These experiments support continuous states under the tested training setup; they do not establish that all explicit chain-of-thought approaches are unsuitable for visual reasoning.

The current thought conditions the DiT, and each reasoning round has a corresponding intermediate visual target, such as a segment of the correct maze path. Conditional flow matching teaches different states to represent different stages of a solution. Two time axes must remain distinct: reasoning rounds indicate progress toward a solution, whereas internal diffusion time indicates progress from noise to an image. A thought update does not necessarily occur at every internal denoising step.

2. Terminal Thought Grounding: constrain continuous-state drift with a correct textual solution

Visual supervision alone can make an output locally resemble an answer without requiring its hidden state to preserve the full logical meaning. The authors encode the correct textual reasoning steps together with the original task input to obtain a reference thought, then use an L2 semantic loss to pull the final thought toward it. This constraint applies only at the terminal step: intermediate states can explore incrementally, while the endpoint must converge toward the semantics of a correct answer.

The textual solution is therefore training supervision, not an answer supplied at inference or a prompt produced by an external solver. Terminal-only grounding also avoids forcing each continuous state to match an individually specified textual sentence. The method still needs reliable textual solutions and intermediate visual targets: latent reasoning does not mean the absence of process supervision.

Some symbols are missing from the locally extracted PDF equations. The following is a schematic reconstruction from the prose describing terminal L2 alignment and its weight, rather than a verbatim recovery of the original equation:

\[ \mathcal L_{\mathrm{total}}^{\tau} =\mathcal L_{\mathrm{FM}}^{\tau} +\mathbf 1_{\{\tau=T\}}\lambda_{\mathrm{align}} \lVert h_T-h_{\mathrm{ref}}\rVert_2^2, \qquad \lambda_{\mathrm{align}}=1. \]

Here, \(T\) is the final reasoning round, \(h_T\) is the terminal thought reached by the model's own recurrence, and \(h_{\mathrm{ref}}\) comes from the correct textual solution; \(\mathcal L_{\mathrm{FM}}^{\tau}\) is the flow-matching loss for that round's visual target. The important feature is the terminal indicator, rather than imposing the same answer representation on all intermediate thoughts.

3. Progressive Training: learn a solution trajectory before consolidating the final output

Exploring partial solutions and producing a clean final answer are not identical objectives. The first stage supervises visual targets at every reasoning round and adds semantic grounding at the endpoint, encouraging coherent solution progress. Final-image supervision alone might leave the recurrent states with little incentive to perform meaningful intermediate computation.

The second stage retains the intermediate forward reasoning process but does not propagate gradients through intermediate steps; optimization focuses on the final output and terminal alignment. The authors use a short second-stage fine-tuning cycle to limit damage to the learned trajectory. This changes training gradient paths, rather than deleting intermediate thoughts or running only the last encoding step at inference.

Both stages rely on coordination between the encoder and DiT: the former constructs useful constraint representations, and the latter realizes them spatially. Adapting only one side performs substantially worse, but the main text provides no separate numerical ablation removing the second stage. The system's entire improvement over a baseline therefore cannot be attributed to progressive training alone.

A Worked Example¶

Consider a maze image whose training data include the correct route and rendered targets for its segments. Early thought states correspond to partial routes, while later states correspond to increasingly complete paths. DiT flow-matching supervision teaches these states to produce their associated images. The terminal thought must also approach a reference derived from the correct direction sequence and original maze, discouraging a locally plausible but globally incorrect route.

At deployment, the same maze and instruction form a fixed prefix. The model recursively updates only its thought states and generates the final route image after exhausting the budget. In the paper's Maze-32 budget experiment, accuracy is 11% with 2 steps and 90% with 50 steps. These are test-set statistics at the respective budgets, not the successful trajectory of an individual example. Intermediate images can help inspect progress but are not a mandatory cost of every inference run.

Loss & Training¶

Flow matching learns a vector field from noise to a target image, with each round using its intermediate or terminal visual target. The first stage aggregates visual losses over all rounds and adds terminal grounding. The second stage retains only the final round's optimization signal. The terminal alignment weight above is 1.

The main text reports a LoRA rank of 32, a learning rate of \(1\times10^{-4}\), 5 training epochs, and task images rendered at \(512\times512\). The current cache does not include the referenced hyperparameter appendix, so the allocation of iterations between stages, the complete LoRA target-layer list, and the hardware cannot be established from it and should not be guessed.

Key Experimental Results¶

Main Results¶

The paper distinguishes zero-shot, task-specific training, and unified training. The table compares EndoCoT with DiffThinker only within the same training protocol. Higher accuracy is better, and differences are in percentage points. Maze suffixes denote maze sizes, and the TSP headers distinguish problem scales. Sudoku suffixes are used as difficulty scales in the main table, but the cache does not explicitly define the counted quantity, so they are not interpreted here as numbers of empty cells. Avg follows the reported aggregate without assuming an unspecified weighting scheme.

Protocol and metric	EndoCoT accuracy (%) ↑	DiffThinker accuracy (%) ↑	Difference (percentage points)
Task-specific / Avg	92.1	83.8	+8.3
Task-specific / Maze-32	90	65	+25
Task-specific / TSP-18	73	59	+14
Task-specific / Sudoku-35	95	55	+40
Task-specific / Sudoku-30	64	13	+51
Unified training / Avg	84.2	77.1	+7.1
Unified training / Maze-32	52	66	-14

An average improvement therefore does not mean superiority in every setting. EndoCoT is worse than DiffThinker on Maze-32 under unified training, and task-specific models are a different deployment condition from one model covering all tasks.

Ablation Study¶

The following rows come from Tables 2 and 3 and select Maze-32 throughout. Path Repetition is the overlap ratio between generated and ground-truth paths, with higher values being better; it does not count erroneous loops. The text does not specify the denominator or exact grid-matching rules, so the reported values are preserved without inventing a precise implementation formula.

Config	Accuracy (%) ↑	Path Repetition (%) ↑	Note
Full model	90	98.13	Continuous thoughts and terminal semantic supervision
Without semantic loss	14	67.24	No terminal textual grounding
Explicit tokens	0	0.08	Discrete autoregressive text replaces continuous thoughts
MLLM-Only	0	0.20	Ablation optimizing only the multimodal language side
DiT-Only	18	80.96	Ablation optimizing only the DiT side

Removing semantic loss reduces accuracy by 76 percentage points, showing strong dependence on textual grounding in this setup. DiT-Only retains 80.96% path overlap but achieves only 18% accuracy, illustrating the gap between matching most path positions and producing a valid complete solution.

The third table retains the reasoning-budget analysis. Time is in seconds and lower is better. These values come from Table 4 and are not mixed with the DiffThinker results in Table 1, because the paper does not explain why the baseline results differ between the tables.

Config	Maze-32 accuracy (%) ↑	Path Repetition (%) ↑	Time (seconds) ↓
DiffThinker	56	73.74	15.72
EndoCoT, 2 steps	11	45.26	16.02
EndoCoT, 5 steps	27	63.90	16.54
EndoCoT, 10 steps	49	82.33	17.52
EndoCoT, 20 steps	74	96.47	19.27
EndoCoT, 50 steps	90	98.13	24.81

Key Findings¶

A larger latent reasoning budget helps difficult mazes but is not a free speedup: low-budget configurations can underperform DiffThinker, and high accuracy takes additional time.
Terminal textual grounding and joint adaptation have strong ablation support. Progressive training lacks a separate quantitative ablation and should be assigned a different strength of evidence.
Several standard VSP scores are near saturation, making difficult mazes and Sudoku more informative. Image-editing results are primarily qualitative and do not establish comprehensive superiority in general editing quality.

Highlights & Insights¶

Separate reasoning depth from denoising depth. Additional computation mainly increases continuous thought updates without requiring expensive DiT decoding at every round, offering a budget-control mechanism distinct from adding sampling steps.
Give visual process supervision and terminal textual supervision distinct roles. The former connects states to executable spatial progress, while the latter constrains the final logical endpoint, rather than asking a single pixel loss to do everything.
Report solution validity alongside path similarity. This exposes cases in which an image looks almost correct but still violates constraints, avoiding an assessment of reasoning based solely on visual closeness.

Limitations & Future Work¶

The authors explicitly acknowledge manual selection of reasoning depth and dependence on high-quality intermediate supervision. Automatic depth control and broader task transfer remain unvalidated directions.
The strongest evidence concerns structured visual puzzles. Unified training does not win on every subtask, and task-specific gains should not be extrapolated directly to general visual reasoning.
Some cached equations have corrupted symbols, appendix details are absent, and the exact Path Repetition implementation and baseline differences between Tables 1 and 4 are insufficiently explained. These gaps affect independent reproduction and fine-grained cost comparisons.

Compared with DiffThinker: both investigate visual problem solving through diffusion generation. EndoCoT emphasizes recursive updates of continuous encoder states, supported by terminal textual grounding, rather than relying solely on generation under static conditioning.
Compared with continuous latent reasoning methods: this work brings hidden-state feedback into conditional visual generation, where DiT spatial grounding and intermediate visual supervision are also required. Findings from text-only reasoning cannot simply be assumed to transfer.
Compared with temporal editing methods such as ChronoEdit: video priors can produce smooth state transitions, but continuous change does not automatically satisfy discrete logical constraints. Mazes and Sudoku provide EndoCoT with more direct constraint-based evaluations.

Rating¶

Novelty: 4/5 — Organizes continuous latent reasoning, terminal semantic grounding, and diffusion visual supervision into a concrete training and inference framework.
Experimental Thoroughness: 4/5 — Includes multiple tasks, unified training, and important ablations, but lacks an isolated progressive-training ablation and substantial general-task evidence.
Writing Quality: 3/5 — Clearly connects two bottlenecks to the design, while some training/inference descriptions and table protocols need sharper distinctions.
Value: 4/5 — Provides a concrete approach to studying reasoning budgets in diffusion models, with applicability still limited by process supervision and task structure.