DARL: Efficient Document-to-Markup Generation via Look-Ahead Diffusion Trajectory Sampling

Conference: ECCV2026
Paper: Official paper page · PDF
Code: https://github.com/SCUT-DLVCLab/DARL
Area: VLM Efficiency / Document Parsing
Keywords: Document-to-markup generation, sliding diffusion blocks, online trajectory sampling, look-ahead rewards, preference optimization

TL;DR

DARL preserves a verifiable autoregressive prefix while refining candidate tokens in a short parallel window, then uses online sampling and look-ahead rewards to learn trajectories that are easier to accept consecutively, achieving a 2.31x speedup and an overall score of 87.69 with dots.ocr-3B on OmniDocBench-1.5 without improving every recognition submetric over the original AR model.

Background & Motivation

Converting scanned pages into Markdown, LaTeX, or HTML requires more than character recognition: the model must recover table-cell boundaries, matching formula brackets, and reading order. Autoregressive models condition each new token on the generated prefix, which suits these tightly coupled dependencies, but long documents require many sequential forward passes. In this paper, the dots.ocr baseline averages 1091.5 forward passes and 25.85 seconds per page on OmniDocBench-1.5, making output length an important inference bottleneck.

Predicting an entire text segment in parallel is faster but can produce mutually inconsistent structural symbols. Fixed-step diffusion also has a stopping problem: too few iterations leave errors unresolved, while too many waste computation. Methods such as LADE reduce sequential steps through candidate verification, but simple candidate priors may be poorly suited to dense formulas and tables. Training introduces a deeper mismatch: inference exposes the model to its own unfinished predictions, whereas clean histories or static noise do not fully teach it to recover from the errors it actually makes.

DARL therefore shifts the acceleration target from individual token predictions to candidate trajectories that can be accepted consecutively. Short windows provide parallelism, verification determines what can be committed, online sampling exposes current-model errors, and rewards evaluate both immediate prefix correctness and subsequent predictability. Core idea: retain AR-style verification as the commitment boundary, use diffusion to propose and refine candidates, and train more efficient trajectories with immediate and look-ahead rewards aligned with consecutive acceptance.

Method

Overall Architecture

The input is a document image and its parsing task; the output is structured markup text. The main experiments initialize from dots.ocr-3B. During inference, a Sliding Diffusion Block (SDB) separates the confirmed prefix from an unconfirmed window. During training, Online Monte Carlo Trajectory Generation (OMTG) supplies candidates and reconstruction supervision, while Diffusion Trajectory Preference Optimization (DTPO) adjusts their probabilities using immediate and look-ahead rewards.

The diagram follows the dependency from the decoding mechanism to matching trajectory sampling and then trajectory optimization. OMTG and DTPO belong to the training loop only: deployment does not require target text or reward computation. The updated model continues to decode through SDB.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    Input["Document image and stable prefix"] --> SDB["Sliding Diffusion Block SDB<br/>Parallel refinement and verification"]
    SDB -->|Commit and slide at inference| Output["Structured markup text"]
    SDB -->|Sample candidates during training| OMTG["Online Trajectory Generation OMTG<br/>Current-model noise and reconstruction"]
    Target["Training target text"] --> OMTG
    OMTG --> DTPO["Trajectory Preference Optimization DTPO<br/>Immediate and look-ahead rewards"]
    DTPO -->|Joint model update| SDB

Key Designs

1. Sliding Diffusion Block SDB: speculate in parallel, but commit only the consecutively verified portion

The model receives a stable prefix and a speculative window of length \(w\), with \(w=16\) in the main configuration. The window contains both unconfirmed candidate tokens and <INIT> initialization placeholders. A forward pass performs two operations together: AR-style next-token prediction immediately after the stable prefix, and parallel refinement of later candidates and placeholder positions. Here, diffusion means iterative refinement of discrete token trajectories, not adding continuous Gaussian noise to the document image or committing an entire window at once.

Figure 3 in the paper identifies a consecutively acceptable segment from left to right by checking consistency between the current results and the previous speculative trajectory; acceptance stops at the first disagreement. Verified positions join the stable prefix, unconfirmed positions remain subject to refinement, and the window slides forward with new initialization tokens. Actual verification results therefore govern denoising progress instead of forcing every window through the same number of refinement steps. Even an individually plausible later token cannot be committed across an earlier disagreement because its conditioning context is not yet reliable.

The guarantee needs a careful boundary. The authors claim that verification preserves the AR distribution, but agreement with the same model's AR behavior is not equivalent to agreement with the true document or guaranteed markup validity; it is also not equivalence to dots.ocr before fine-tuning. The cached main text does not fully specify stochastic acceptance probabilities, attention masks, or cache implementation, so the available description does not support reconstructing a rigorous distribution-equivalence proof.

2. Online Trajectory Generation OMTG: train on the current model's own errors

At a selected offset in the target sequence, training splits a window into a candidate segment and an initialization segment of lengths \(l_p\) and \(l_n\), with \(l_p+l_n=w\). The paper gives the candidate-length range as \(0\) to \(w-1\). With gradients disabled for sampling, the model uses temperature-scaled multinomial sampling conditioned on the stable context to produce \(K=4\) candidate trajectories, then appends \(l_n\) <INIT> tokens to each. Temperature sampling explores different candidates, but the cached main text does not report the temperature value.

These composite inputs are fed back into the model to reconstruct the corresponding ground-truth tokens. Unlike common denoising objectives that supervise only masked positions, DAR supervision extends from the sequence start through the current window boundary: the model must repair speculation while preserving basic AR modeling. Because candidates come from the model being updated, the error distribution evolves during training instead of remaining fixed to preconstructed noise. This mitigates the mismatch inside speculative windows; the stable prefix still comes from the target sequence, so it does not establish elimination of all long-range exposure bias.

3. Trajectory Preference Optimization DTPO: reward trajectories that help both current and future acceptance

Cross-entropy evaluates predictions position by position, whereas decoding efficiency depends on how many tokens are correct consecutively from the left. DARL therefore uses the longest common prefix (LCP), rather than whole-segment similarity alone. The immediate reward \(r_1\) is the LCP fraction between a candidate and its corresponding target segment. For the look-ahead reward \(r_2\), the model receives that candidate followed by a new initialization segment, predicts the subsequent \(l_n\) tokens, and measures their LCP fraction:

\[ r_1(\hat{h})=\frac{|\operatorname{LCP}(\hat{h},y_{\mathrm{target}})|}{|y_{\mathrm{target}}|},\qquad r_2(\hat{h})=\frac{|\operatorname{LCP}(\hat{y}_{\mathrm{future}},y_{\mathrm{gt\_future}})|}{l_n},\qquad R=r_1+r_2. \]

The two rewards serve different purposes. A consecutively correct candidate does not by itself establish that the model can easily predict what follows when conditioned on it; the look-ahead term tests that with another prediction. It is neither a full-document syntax checker nor planning through the sequence end, but a local future-window proxy. LCP is chosen not because it is more comprehensive, but because it matches the decoding rule that stops commitment at the first error.

Total rewards are normalized using the mean and standard deviation within the candidate group to obtain advantages:

\[ A_i=\frac{R_i-\mu_R}{\sigma_R+\epsilon}. \]

Above-average trajectories receive positive updates, while below-average trajectories have their probabilities reduced. The paper describes this as a GRPO-inspired approach without a separate value network. Figure 5 additionally identifies DAR supervision as a persistent anchor that avoids a separate reference model. Cached Equation (7) presents an advantage-weighted token log-probability objective; the GRPO label is not a basis for adding PPO clipping or KL terms absent from the stated formulation.

A Worked Example

This is an explanatory example, not an experimental sample reported in the paper. Suppose a document formula is being rendered with a subscript and closing brackets. Use the main SDB setting \(w=16\), and split a training window into \(l_p=8\) and \(l_n=8\), sampling \(K=4\) candidates. One candidate matches its target for the first 6 tokens but makes a bracket error at token 7. Its immediate reward is \(6/8=0.75\); a coincidentally correct token 8 does not extend the LCP.

Next, use that candidate as speculative context to predict the following 8 tokens. If the first 4 future tokens are consecutively correct, the look-ahead reward is \(4/8=0.5\), giving a total reward of \(1.25\). Whether its advantage is positive depends on the other 3 candidates. The example illustrates that DTPO scores a trajectory's contribution to current and future consecutive correctness, rather than treating its reward as an inference-time acceptance probability.

At deployment, no ground-truth target is available for computing LCP rewards; only SDB's model-consistency verification is used. Rewards train a better candidate distribution, whereas verification decides which tokens can actually be committed during decoding.

Loss & Training

The total objective combines DAR reconstruction supervision and DTPO preference optimization with weights of 1 and 0.1:

\[ \mathcal{L}_{\mathrm{Total}}=\lambda_1\mathcal{L}_{\mathrm{DAR}}+\lambda_2\mathcal{L}_{\mathrm{DTPO}},\qquad \lambda_1=1,\quad\lambda_2=0.1. \]

The training corpus contains 49K documents spanning magazines, newspapers, handwritten materials, examination papers, textbooks, slides, financial research reports, and scanned old books. PaddleOCR-VL provides initial annotations; image-related non-textual tokens are removed, and layout and recognition errors are corrected manually. This curation can improve accuracy independently of decoding, making the same-data fine-tuned AR control important.

The main model trains for 2 epochs on 8 NVIDIA A800 GPUs, with a per-device batch size of 1, taking approximately 5 days. It uses AdamW, a peak learning rate of \(2\times10^{-5}\), 500 warmup steps, and weight decay of 0.01. Evaluation uses Flash-Attention 2. MSN and EAGLE-3 are adapted to the same dots.ocr backbone and trained on the curated data, while Jacobi and LADE use their official training-free decoding implementations.

PDF extraction has damaged operators and subscripts in some cached equations. The rewards, advantages, and total objective above are presented according to the meanings explicitly supported by adjacent prose. Exact DAR indexing, the complete DTPO implementation, and unreported sampling details are not invented; reproduction should consult the original PDF and code. The repository URL comes from the paper, and its contents were not checked online for this note.

Key Experimental Results

Main Results

The following selection comes from Table 1 of the paper and uses dots.ocr-3B throughout. Time is seconds per document, and iterations are average forward-pass counts. Higher Overall, Table, and TableS are better; Text is edit distance, for which lower is better. The asterisk denotes an AR baseline further fine-tuned on the curated data. All speedups are relative to the original AR row.

OmniDocBench-1.5 configuration	Speedup	Time	Iterations	Overall	Text	Table	TableS
dots.ocr AR	1.00×	25.85	1091.5	87.57	0.065	83.76	87.39
MSN	2.07×	12.49	527.2	87.23	0.093	85.81	88.76
EAGLE-3	1.74×	14.88	625.0	87.38	0.084	84.40	88.21
dots.ocr AR*	1.04×	24.86	1033.3	87.15	0.095	83.35	85.69
DARL	2.31×	11.19	485.6	87.69	0.101	87.73	89.41

DARL gains 0.12 overall points and 3.97 Table points over the original AR model, but Text worsens from 0.065 to 0.101; formula CDM also slightly declines from 85.44 to 85.43. It gains 0.54 overall points over the same-data AR* baseline, but this cannot attribute the entire improvement to any single reward. The abstract's claim of matching or exceeding accuracy should be read as an aggregate result, not losslessness in every element category.

The next selection comes from Table 2. olmOCR-Bench evaluates document reconstruction through fine-grained unit tests. The reported Overall values retain their original plus/minus notation, but the cached main text does not define its statistical interpretation, so it is not relabeled as a standard deviation or a particular confidence level.

olmOCR-Bench configuration	Speedup	Time	Iterations	Overall	Old Scans	Headers & Footers	Long Tiny Text
dots.ocr AR	1.00×	28.65	1216.9	78.9 ± 1.0	41.6	85.9	79.2
MSN	1.98×	14.46	614.1	77.9 ± 1.1	53.8	90.4	70.6
dots.ocr AR*	1.08×	26.52	1122.3	79.9 ± 1.0	57.0	97.2	75.1
DARL	2.27×	12.61	519.3	81.7 ± 1.0	62.5	89.2	74.7

The overall gain is 2.8 points over the original AR model and 1.8 over AR. Old-scan improvements partly reflect data curation, already visible in AR reaching 57.0, with DARL improving further to 62.5. However, its Headers & Footers score of 89.2 is below MSN's 90.4 and AR*'s 97.2, while Long Tiny Text at 74.7 trails the original AR score of 79.2. These results do not establish dominance across all complex layouts.

Ablation Study

The AR row and the three SDB variants below come from Table 3; the final row comes from the reward replacement experiment in Table 4. The ED row is a separate control, not another cumulative component added after the preceding rows.

OmniDocBench-1.5 configuration	Speedup	Time	Iterations	Overall
AR, no SDB or rewards	1.00×	25.85	1091.5	87.57
SDB, no immediate/look-ahead rewards	1.98×	13.03	547.3	87.21
SDB + immediate reward	2.14×	12.09	519.7	87.53
SDB + immediate + look-ahead rewards, LCP	2.31×	11.19	485.6	87.69
DARL, ED instead of LCP	2.13×	12.14	522.4	87.85

Key Findings

• SDB provides most of the initial acceleration but loses 0.36 overall points. The immediate reward recovers 0.32 points, and adding look-ahead improves another 0.16 points while reducing time from 12.09 to 11.19 seconds. Look-ahead improves both speed and overall accuracy, but its accuracy increment is not larger than the immediate reward's increment.
• ED achieves a higher score of 87.85 than LCP's 87.69; LCP instead offers a 2.31x rather than 2.13x speedup. This supports reward alignment with the acceptance mechanism, not superiority of LCP on every evaluation axis.
• For window sizes \(w=8,16,24,32\), Overall scores are 87.53, 87.69, 82.91, and 82.74. Larger windows substantially damage accuracy. Figure 6's time annotations differ in value or reporting convention from the main table, so those timings are not mixed into the main results.
• Scaling InternVL3.5 across 2B, 8B, and 14B yields speedups of 2.20, 2.52, and 2.78x, respectively, but Overall decreases from 80.76, 86.81, and 92.67 to 80.19, 86.28, and 92.50. The 14B result of 2.78x is a scaling experiment, distinct from the main model's 2.31x.

Highlights & Insights

• The reward unit closely tracks actual system savings. Correct tokens after the first error may improve edit distance without extending the segment that can be committed in that step; LCP captures this distinction.
• Look-ahead goes beyond whether a candidate appears correct to whether it provides useful predictive context. The idea could be explored for code completion or GUI-to-HTML generation, but these transfers are not evaluated in the paper.
• Online noise and the supervised anchor are complementary. The former teaches recovery from current-policy errors, while the latter prevents trajectory rewards from becoming the only training signal, preserving document recognition and AR modeling foundations.

Limitations & Future Work

• The authors explicitly leave open-ended tasks such as image captioning unverified. When multiple outputs are equally valid, strict prefix matching can assign low rewards to useful candidates. Structurally or semantically equivalent rewards merit study, but still need a connection to the acceptance mechanism.
• Evaluation reaches only 14B parameters and does not establish benefits beyond 70B. Window size, candidate count, and denoising ability may change with scale; the existing trend is insufficient to establish a universal scaling law.
• Assessment in this note: curated data and decoding-oriented training both change the model, so inference acceleration must be distinguished from recognition improvement. The component ablation does not separately replace OMTG with static trajectories, leaving the independent contribution of online sampling unquantified.
• Assessment in this note: table-structure gains coexist with text-submetric regressions, and claims of absolute syntactic correctness cannot replace error analysis. The main text does not break down bracket failures or cross-window table misalignment, nor fully report acceleration across deployment hardware, batch sizes, and sampling randomness.
• Reproduction has information limits: cached equations contain extraction damage, and the sampling temperature, zero-length candidate handling, and full verification implementation need confirmation from source code. Tree-structured speculative verification is proposed as future work, not an implemented component credited in this note.

Related Work & Insights

• Versus LADE / Jacobi decoding: these methods reduce sequential dependencies through parallel iteration and candidate verification. DARL additionally trains candidate trajectories, using current-model online sampling and look-ahead rewards to improve acceptability, at the cost of additional training.
• Versus MSN / EAGLE-3: MSN uses noisy training to support parallel inference, while EAGLE-3 follows a speculative acceleration approach. Both are adapted to the same document backbone here. DARL's distinction is to train directly for current and future consecutive correctness of window trajectories, rather than evaluating candidate-generation speed alone.
• Versus DiffusionVL / SDAR-VL: these methods explore combinations of AR models and diffusion-based vision-language generation. DARL emphasizes sliding verification to determine the committable region and tailors training to that execution pattern. Its demonstrated advantage is bounded by document-parsing evaluations, not universal superiority over diffusion VLMs.
• Versus GRPO-style preference optimization: the optimization target is a decoding trajectory, rather than response style or a reasoning answer. A reusable lesson is to identify the condition under which the inference engine actually saves computation, then design rewards that improve that condition.

Rating

• Novelty: 4/5. Combining sliding verification, online trajectories, and local look-ahead rewards targets structured document acceleration, although the underlying mechanisms have precedents.
• Experimental Thoroughness: 4/5. Two benchmarks, same-data AR controls, component and reward ablations, and backbone and scale studies provide substantial coverage; isolated OMTG controls and broader deployment conditions remain missing.
• Writing Quality: 3/5. The framework and tables are clear, but absolute-correctness claims and some category-level claims exceed the evidence, while equation extraction damage limits readability of the cached version.
• Value: 4/5. More than 2x measured acceleration in the main configuration has practical value when the training cost is affordable and some text-recognition regressions are acceptable.