MinerU-Diffusion: Rethinking Document OCR as Inverse Rendering via Diffusion Decoding

Conference: ECCV2026
Paper: ECCV Paper
Area: Multimodal VLM
Keywords: document OCR, masked diffusion, block attention, confidence scheduling, curriculum learning

TL;DR

MinerU-Diffusion treats document OCR as inverse rendering from images to structured symbols, combining autoregression across blocks, diffusion within blocks, and a two-stage curriculum to reach 93.37 Overall on OmniDocBench v1.5 with ground-truth layout and approximately 2.1 times faster decoding at its default threshold.

Background & Motivation

Modern document OCR must recover reading order, table boundaries, and mathematical expressions, not merely transcribe characters. Vision-language systems such as MinerU2.5 and PaddleOCR-VL serialize these elements into text, but token-by-token autoregressive decoding requires many sequential forward passes for long documents. More importantly, a language decoder may complete plausible text when visual evidence is unclear instead of faithfully reading the page.

The authors argue that a document already exists on a two-dimensional page: the left-to-right output order is primarily a serialization convention, not the intrinsic causal process that generates its characters. Given the page, many positions can be recognized simultaneously, with tables and formulas constrained mainly by spatial and structural relationships. Any-order diffusion is therefore appealing, but applying full-attention denoising to an entire document introduces computational overhead, positional drift, and repetitive output.

The paper does not remove generation order altogether. It keeps ordering across blocks, permits parallelism within each block, and stabilizes learning by covering diverse documents before refining uncertain cases. Core Idea: recover local OCR symbols through visually conditioned parallel denoising, use block boundaries to constrain long-sequence drift, and let confidence determine which predictions can be committed early.

Method

Overall Architecture

The input consists of a document image and a task prompt. The output is a structured sequence from a shared vocabulary covering text, layout markers, table delimiters, and mathematical operators. Diffusion operates over discrete output tokens, not page pixels: this is neither image denoising nor an image restoration model.

The system follows the remaining MinerU2.5 components, replaces its language decoder with SDAR-1.7B-Chat-b32, removes MRoPE, and conditions decoding on native-scale visual features. The output is partitioned into fixed-length blocks: completed blocks provide context while the current block recovers masked positions over multiple prediction rounds. Training first establishes visual alignment and broad OCR competence, then introduces refined hard examples. The diagram separates inference from the training path that supplies model parameters.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    Input["Document image and prompt"] --> Blocks["Block-Conditional Denoising"]
    Blocks --> Schedule["Confidence-Based Dynamic Scheduling"]
    Schedule --> Output["Structured document sequence"]
    Data["Diverse documents and labels"] --> Curriculum["Two-Stage Uncertainty Curriculum"]
    Curriculum -.->|Train model parameters| Blocks

Key Designs

1. Block-Conditional Denoising: retain cross-block anchors while enabling parallel updates

Denoising an entire document at once allows local mistakes to affect distant structures, particularly through repeated delimiters in long tables. The proposed decoder partitions the sequence into contiguous blocks. Tokens in the current block can access all preceding blocks and the visual input, but not future blocks; attention within the current block is bidirectional, allowing its positions to participate jointly in reconstruction. The central factorization in Equation (5) is:

\[ p_\theta(y\mid x)=\prod_{b=1}^{B}p_\theta\big(y^{(b)}\mid y^{(<b)},x\big). \]

Here \(B\) is the number of blocks, \(y^{(<b)}\) denotes previously generated blocks, and \(x\) is the document image. Each conditional distribution is approximated through multiple discrete denoising steps within the current block. Thus, the method is not fully non-autoregressive at the document level: causal dependencies remain across blocks, while token commitment within a block need not proceed character by character from left to right. Fixed block boundaries anchor positions instead of allowing the entire long sequence to drift at every iteration.

This attention structure also supports KV-cache reuse for completed prefixes, reducing repeated computation. Experiments fix the block length at 32. The paper discusses local attention complexity, but the model still attends to preceding blocks; a local computation expression should not be interpreted directly as an end-to-end complexity or latency guarantee for the complete system.

2. Confidence-Based Dynamic Scheduling: reserve computation for uncertain positions

After a forward prediction on the current block, dynamic scheduling uses confidence to select tokens that can be committed, leaving uncertain positions for subsequent denoising. Unlike static schedules that commit a predetermined number of tokens per round, this allows clear characters to finish quickly while ambiguous symbols and structural boundaries receive more computation. Existing predictions also provide context for other positions in the same block without requiring a strictly growing left prefix.

The inference threshold \(\tau_{\mathrm{decode}}\) controls conservativeness: a lower threshold permits more token commitments per forward pass and higher throughput, but increases the risk of accepting mistakes prematurely. A very high threshold reduces parallelism. The default is 0.95. This threshold does not select training examples and must not be confused with \(\tau_{\mathrm{train}}\) below. The main text does not provide complete fallback pseudocode for the case in which every remaining position falls below the threshold, so this note does not invent that implementation detail.

3. Two-Stage Uncertainty Curriculum: build stable representations before refining difficult boundaries

Diffusion training randomly masks target tokens, making supervision density and available conditional information depend on the mask. Starting directly with noisy, structurally difficult documents can destabilize optimization. Stage 1 uses a curated, diverse dataset with automated annotation refinement, covering different layouts, languages, and document styles to establish general parsing competence. Labels need not be entirely noise-free; broad coverage helps the model learn stable representations.

After Stage 1 converges, the model performs \(T\) stochastic inference passes on an unlabeled or weakly labeled sample and compares their outputs. It uses PageIoU for layout, CDM for formulas, and TEDS for tables. These assess task-specific structural or content agreement rather than applying a single exact-string-match test. Equation (10) averages similarities between all pairs of predictions:

\[ C(x)=\frac{2}{T(T-1)}\sum_{i<j}S\big(\hat y^{(i)},\hat y^{(j)}\big). \]

Here \(S\) is the selected task-specific consistency metric. A low \(C(x)\) indicates unstable predictions for the same image. Samples below the task-dependent threshold \(\tau_{\mathrm{train}}\) enter an AI-assisted human annotation pipeline. Agreement does not guarantee correctness: the score identifies cases worth reviewing rather than replacing ground-truth annotation.

Refined hard examples are mixed with randomly sampled foundational data. The latter helps retain general competence instead of shifting training entirely toward atypical cases. Hard examples also receive uncertainty-dependent weights. Equations (14)-(15) can be written together as:

\[ \mathcal L_{\mathrm{hard}}(\theta)=\mathbb E_{(x,y)\sim\mathcal D_{\mathrm{SFT}}}\left[(1+\beta(1-C(x)))\,\ell(f_\theta(x),y)\right]. \]

The coefficient \(\beta\) controls the additional weight assigned to low-consistency samples. This procedure changes annotation quality, training difficulty, and supervision weights simultaneously; its gains cannot all be attributed to the weighting formula alone.

A Worked Example

Consider a page containing a table whose serialized output has 64 tokens. This is an illustrative walkthrough using the paper's block length, not a measured paper example: the model processes the first 32 positions and then the next 32, conditioning each block on the image and the completed prefix.

Within the first block, clear text and delimiters can be committed together while ambiguous cell content undergoes further denoising. Once that block is complete, its KV cache supports the second block. If repeated stochastic predictions on such pages produce inconsistent table structures, training may select them as low-TEDS-consistency examples for human refinement and Stage 2. Multiple full inference runs are used for training-time mining; deployment does not inherently require recognizing every page \(T\) times.

Loss & Training

Foundational training randomly masks positions in the target sequence and computes prediction loss on those masked positions. Stage 2 applies the sample weights above to a mixture of refined hard examples and random foundational samples. The main text delegates further optimization details to supplementary material, so this note does not supply unsupported learning rates, batch sizes, epoch counts, mixing ratios, or mining thresholds.

All training data originate from the MinerU2.5 dataset, totaling approximately 7.5M samples and primarily covering Chinese and English documents. The model first receives VQA fine-tuning on what the main text calls the LLaVA-NeXT dataset, followed by specialized OCR training. The 1.7B in the SDAR decoder name is not the complete vision-language model size: the main results table lists the full model as 2.5B.

Key Experimental Results

Main Results

Original Table 1, OmniDocBench v1.5. Default dynamic decoding uses block length 32, \(\tau_{\mathrm{decode}}=0.95\), top-k=0, temperature=1.0, and top-p=1.0. Higher Overall, Formula, and Table TEDS are better; lower Text error is better. GT Layout supplies ground-truth layout and must be separated from fully automatic evaluation.

Method	GT Layout	Overall	Text	Formula	Table TEDS
MinerU2.5	No	90.67	0.047	88.46	88.22
PaddleOCR-VL	No	92.56	0.035	91.43	89.76
MinerU-Diffusion	No	88.94	0.061	86.41	86.50
MinerU2.5	Yes	93.44	0.025	91.98	90.84
PaddleOCR-VL	Yes	93.91	0.021	92.13	91.70
MinerU-Diffusion	Yes	93.37	0.028	91.92	91.00

Ground-truth layout raises the proposed model's Overall from 88.94 to 93.37, a 4.43-point difference. Its layout-assisted result approaches MinerU2.5, but fully automatic parsing still trails both specialized systems; this is not evidence of universally state-of-the-art accuracy.

Original Table 2 additionally reports TEDS/TEDS-S of 73.77/82.06 on CC-OCR and 81.18/88.66 on OCRBench v2. UniMER-Test CPE/HWE/SCE/SPE scores are 91.6/91.6/92.0/96.8. TEDS-S emphasizes table structure. Every listed result is below MinerU2.5 in the same table, showing that parallel generation does not eliminate the specialized recognition accuracy gap.

Ablation Study

Original Table 4 evaluates the curriculum. The following retains the table's two-decimal values instead of mixing them with higher-precision numbers elsewhere in the prose.

Training Config	Overall with GT Layout	Overall without GT Layout	Table TEDS with GT Layout
Stage 1	92.89	86.13	90.28
Stage 2	89.33	35.71	83.29
Stage 1 + Stage 2	93.37	88.94	91.00

The full curriculum improves fully automatic Overall by 2.81 points over Stage 1 alone. Stage 2 alone falls to 35.71: difficult-example training requires foundational representations rather than simply maximizing exposure to hard cases.

Original Table 3 compares static and dynamic decoding. TPF is the number of tokens committed per forward pass, and TPS is tokens per second. Its dynamic threshold is 0.97, not the main results' 0.95. Figure 3 specifies NVIDIA H200 hardware and batch size 1 for throughput measurement.

Decoding Strategy	TPF	TPS	Overall with GT Layout
Static, 6 steps	5.33	91.56	88.31
Dynamic, threshold 0.97	5.18	98.32	93.34
Static, 32 steps	1.00	21.86	93.02
MinerU2.5	1.00	51.46	93.44

Key Findings

• Figure 3 and Section 4.4 report 108.9 TPS with accuracy above 93% at threshold 0.95, approximately 2.1 times the roughly 52 TPS MinerU2.5 baseline. Threshold 0.6 reaches 164.8 TPS with accuracy above 90%, producing the approximately 3.2 times peak speedup. This is a different operating point from default accuracy.
• In Table 3, dynamic decoding obtains substantially higher Overall than static 6-step decoding with similar TPF. Which tokens are committed matters more than fixing a parallel token count; static 32-step decoding is accurate but even slower than the AR baseline.
• Semantic Shuffle starts from 112 English FOX document images, shuffles a controlled fraction of words, and re-renders them with comparable formatting. Figure 7 shows greater diffusion robustness to semantic disruption, but the cache has no readable curve values, so no numerical degradation rates are invented here.

Highlights & Insights

• Separating serialized output from mandatory sequential generation is the central modeling insight. Strong visual conditioning makes parallel recovery more plausible in OCR than in open-ended writing.
• Block boundaries serve both as structural anchors and cache boundaries. This connects long-sequence stability to system efficiency rather than merely reducing denoising steps.
• Disrupting linguistic coherence tests whether a model is genuinely reading the image more directly than scores on natural documents alone. Similar controlled evaluations could benefit other image-to-symbol transcription tasks.

Limitations & Future Work

• The authors explicitly note that training focuses on Chinese and English, with no dedicated low-resource-language evaluation. Semantic Shuffle's 112 English pages likewise do not establish broad multilingual robustness.
• The authors identify layout analysis as a major bottleneck: ground-truth layout changes Overall by 4.43 points. Complex formulas and table content also remain less accurate than some specialized AR systems.
• As an experimental-design caveat, the full model has 2.5B parameters versus MinerU2.5's 1.2B. Architecture, curriculum, and annotation refinement change together, preventing a pure causal-order interpretation of accuracy differences.
• The cache contains the main paper, not all supplementary implementation details it references. The full-attention comparison uses LLaDA-MoE-7B-A1B-Instruct rather than the main model and primarily provides examples and qualitative conclusions; precise same-backbone ablation gains should not be invented.
• Future evaluation should include end-to-end latency covering visual encoding, layout parsing, and scheduling, as well as batching and longer documents. Existing TPS measurements do not establish whole-page speedups across all deployment settings.

Related Work & Insights

• vs MinerU2.5: The proposed system reuses its data and remaining architecture components, primarily changing the decoding paradigm and adding curriculum learning. Its advantages concern throughput and semantic-perturbation robustness rather than uniformly better recognition accuracy.
• vs SDAR / Block Diffusion: Causal dependencies across blocks and diffusion within blocks are established foundations. This work adapts them to native-scale visual conditioning and structured document parsing, together with an OCR hard-example curriculum.
• vs Full-Attention Diffusion and Speculative Decoding: The former permits sequence-wide interaction but risks long-sequence drift; the latter accelerates AR verification while retaining left-to-right commitments. The proposed block-level approach sits between globally unordered and token-sequential generation.

Rating

• Novelty: 4/5. Combines block diffusion, the inverse-rendering view of OCR, and a hard-example curriculum, although the basic decoding mechanism is established.
• Experimental Thoroughness: 4/5. Covers pages, tables, formulas, scheduling, and curriculum ablations, but multilingual evidence and isolated mechanism comparisons remain limited.
• Writing Quality: 4/5. Explains the method and accuracy-throughput trade-off clearly, with some implementation and quantitative details deferred to supplementary material.
• Value: 4/5. Offers a practical direction for high-throughput document transcription, subject to layout and recognition accuracy gaps.