ECHO: Efficient Chest X-ray Report Generation with One-step Block Diffusion¶

Conference: ECCV2026
Paper: ECCV Paper
Project: https://echo-midea-airc.github.io/
Area: Medical Imaging
Keywords: chest X-ray report generation, block diffusion, direct conditional distillation, response-asymmetric diffusion, report normalization

TL;DR¶

ECHO converts a medical autoregressive model into a block diffusion model and distills conditional distributions from multi-step teacher trajectories into a one-step-per-block student, retaining strong clinical-content scores on the paper's normalized chest X-ray report test sets while reaching 274.21 tokens/s with block size 8.

Background & Motivation¶

Chest X-ray report generation combines image understanding, abnormality localization, and medical language production; fluent text can still be wrong, particularly when laterality, negation, and finding names must agree. Most medical vision-language models generate autoregressively, one token at a time, so report decoding remains sequential after image encoding and limits high-throughput workloads. Discrete diffusion models can predict several masked positions in parallel, but usually require repeated denoising: confident words are committed first and then help determine the remaining words. Parallel prediction therefore does not automatically mean a single model forward pass.

Filling every position in a block at once can produce a fragment whose individual words are plausible but whose combination is incoherent. The paper attributes this to mean-field bias: a denoiser approximates a joint distribution with per-position distributions and may combine words belonging to different plausible descriptions. Conventional token-wise distillation, even with cleaner teacher states, does not necessarily transfer the dependencies progressively established during multi-step generation to a one-step student. Meanwhile, clinical reports often mention only abnormalities and omit normal structures, leaving supervision ambiguous between an unmentioned finding and an explicitly absent abnormality.

ECHO therefore changes more than the sampler: it addresses report supervision, diffusion adaptation, one-step distillation, and cache overhead together. The foundation model first learns more complete chest X-ray descriptions; expensive visual context is then retained as a shared condition, and the student learns reliable teacher decisions collected at different denoising steps. One-step always refers to a fixed-length block: the report is still generated sequentially across blocks, not in one pass for the entire report. Core Idea: stitch together teacher distributions obtained after progressively committing context, train a student to predict a fully masked block directly, and use explicit termination supervision and cache fusion to prevent other bottlenecks from eroding quality and speed gains.

Method¶

Overall Architecture¶

The inputs are a chest X-ray image and a report-generation instruction; the output is a natural-language report describing imaging findings. A vision encoder and projector turn the image into visual tokens, which condition the language model alongside the instruction; the main changes concern report training and decoding. Training starts from Lingshu-7B, produces ECHO-AR through continued pretraining on normalized reports, converts it into multi-step ECHO-Base with response-asymmetric diffusion adaptation, and obtains ECHO through direct conditional distillation. At inference time, completed report blocks provide historical context, while the current block starts fully masked and receives parallel token predictions in one forward pass. Fused block KV caching updates the preceding block's cache inside the current block's denoising forward, avoiding a separate model call for cache maintenance. The solid arrows below distinguish training-artifact progression from the decoding path; the dashed arrow represents teacher supervision used only during distillation.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Paired chest X-rays and reports"] --> B["Report Normalization<br/>Continued pretraining"]
    B --> C["Response-Asymmetric Diffusion<br/>RAD adaptation"]
    C --> D["Direct Conditional Distillation<br/>DCD"]
    C -.->|Multi-step teacher trajectory supervision| D
    D -->|Training artifact| E["One-step ECHO weights"]
    E --> F["Fused Block KV Cache"]
    G["Inference: image, instruction,<br/>past blocks and masked block"] --> F
    F --> H["Current block output<br/>Next block or termination"]

Key Designs¶

1. Report Normalization: turn omitted normal states into explicit supervision

Clinical reports commonly prioritize abnormalities; an anatomical region absent from the text is not absent from the image, nor does its omission directly supply an explicit normal label. ECHO rewrites training reports to cover predefined anatomical regions, assigning each region a positive finding or a negative assertion, and uses this corpus for continued pretraining of Lingshu-7B. The model consequently learns both how to describe abnormalities and when to state that no abnormality is observed, reducing ambiguity caused by omission. The resulting ECHO-AR is still autoregressive: normalization improves content learning first and does not itself enable within-block parallelism. The authors interpret its benefit as reducing false-positive hallucinations and false-negative omissions; Table 3 shows that this data choice continues to affect adaptation and distillation.

Adding explicit labels must be distinguished from recovering ground truth from the image: rewriting a report cannot prove that every unmentioned region is normal. Normalization reliability therefore depends on the region rules, rewriting quality, and medical review, none of which can be assessed from aggregate scores alone. The main text does not provide enough detail to reconstruct the complete normalization procedure, and its relevant supplementary references remain unresolved placeholders. Normalization is thus both an important method component and a central issue for reproducibility and clinical interpretation.

2. Response-Asymmetric Diffusion: keep only one copy of long visual context

Converting an autoregressive model into block diffusion requires training representations for both clean historical responses and noisy target responses. Earlier adaptation methods duplicate the entire sequence, including vision, instruction, and response tokens, but long chest X-ray visual sequences make this duplication expensive for attention. RAD duplicates only the response, shares the vision and instruction context, and uses a block attention mask to control what each noisy response block can observe. The current target block can attend to all visual and instruction tokens and the clean content of previous blocks; future responses must not leak into its historical context. Training history comes from a teacher-forcing construction, whereas inference history comes from generated blocks; these stages should not be conflated.

RAD consolidates the previous two-stage conversion into a single supervised fine-tuning stage, trains the language-model backbone while freezing the vision encoder and projector, and produces ECHO-Base. This model still fills the current block through confidence-guided multi-step denoising, making it the distillation teacher rather than the final one-step model. The design targets long visual contexts with relatively short responses, so its training savings cannot be transferred unconditionally to text-only inputs. Figure 5 also shows quality approaching or exceeding ECHO-AR after about 60 training steps while throughput continues improving, separating recovery of semantic competence from maturation of within-block parallel decoding.

3. Direct Conditional Distillation: learn predictions made after the teacher commits context

For each block, DCD first runs the teacher's multi-step denoising process instead of constructing an artificial trajectory unrelated to its actual sampling policy. At each step, the teacher predicts distributions for unresolved positions, treats the largest token probability as confidence, and commits predictions above a threshold. If no position passes the threshold, it commits the most confident position so that block generation continues making progress. When a position is committed, the algorithm records both its predicted distribution and its discrete pseudo-label; subsequent teacher predictions are conditioned on the words already committed. The target for a later position therefore comes from richer context rather than merely its initial marginal guess under complete masking.

After collecting the trajectory, the algorithm constructs a RAD-style block teacher-forcing sequence from the pseudo-labels and trains the student to predict the current block in one pass. For each position, the student's distribution is aligned by forward KL divergence with the distribution recorded when the teacher committed that position, aggregating supervision across blocks and positions. The following equation captures the unweighted base objective in Algorithm 1; the text subsequently adds denoising-step weighting but does not provide its complete normalization formula in the supplied main paper.

\[ \mathcal{L}_{\mathrm{DCD}}=\sum_{n,i}D_{\mathrm{KL}}\!\left(P_{\mathrm{tch}}^{(n)}[i]\,\middle\|\,p_{\phi}(b_{n,i}\mid x_{\mathrm{train}})\right). \]

Here \(n\) indexes a block, \(i\) indexes a position, \(P_{\mathrm{tch}}^{(n)}[i]\) is the teacher distribution recorded upon commitment, and \(x_{\mathrm{train}}\) contains shared conditions and the history visible under RAD. The student and teacher share the image, instruction, and past blocks, but the one-step student does not receive the words progressively supplied within the current block before making its prediction. Thus, matching conditions does not mean that the student observes exactly the same within-block state as every teacher denoising step; the predictive benefit of those additional conditions is precisely what must be distilled. Step-wise weighting strengthens supervision for positions committed later, which the authors associate with stronger token dependencies, while early commitments provide comparatively stable constraints.

The end-of-sequence token <eos> needs separate treatment: the paper finds it less confident than content tokens and increasingly unstable with larger blocks. Imitating only its comparatively flat soft target can leave the student repeating text without terminating, so DCD adds cross-entropy supervision toward a one-hot termination target. This term and step-wise weighting belong to DCD rather than an external inference-time report repair module; termination supervision produces substantial gains in Table 2. The paper calls the trajectory-collected supervision an unfactorized target, but the algorithm still sums per-position KL terms, and the student output head does not become an explicit joint-distribution model. A cautious interpretation is therefore that conditional teacher trajectories empirically mitigate one-step inconsistency, not that they mathematically remove every mean-field representational limitation.

4. Fused Block KV Cache: avoid an extra forward pass just to update history

Standard block KV caching performs another forward pass after completing a block to cache the newly generated content before decoding the next block. This overhead is comparatively small when each block requires many denoising steps; after compression to one step, however, it can turn one model call per block into two. ECHO defers the preceding block's cache update and computes and caches its keys and values in the same forward pass that predicts the current fully masked block. This retains historical conditioning while removing the dedicated cache-update call, allowing distillation's algorithmic savings to translate more directly into practical throughput. The change concerns cache execution order; it does not make sequential report blocks independent parallel outputs.

The authors claim that fusion introduces no additional FLOPs and halves the corresponding forward-call count, but the proof points to an unresolved supplementary section. This note therefore treats it as an implementation strategy described in the main paper rather than an independently verified computational equivalence. Even with fewer calls, end-to-end gains depend on visual encoding, prefill, device utilization, and output length, requiring measured TPS and latency rather than TPF alone.

A Worked Example¶

Consider an illustrative block of length 4 whose four positions are initially masked; this is not an additional experimental case reported by the paper. The teacher might commit two confident positions first, use those words to resolve a third, and then determine the fourth, recording each position's full probability distribution at commitment. These targets preserve how earlier decisions constrain later wording instead of treating four initial independent guesses as the final answer. During training, the student receives the same chest X-ray, instruction, and past blocks but must produce all four predictions from a fully masked current block in one pass. If a target position is <eos>, it receives an explicit termination label in addition to soft-distribution distillation. Deployment no longer reruns the teacher trajectory for this block: the student outputs the block and either continues to the next one or stops at termination. When generation continues, the preceding block's KV update is fused with the next block prediction instead of requiring a separate maintenance call. This explains the throughput distinction between block sizes 4 and 8 while showing that one-step decoding removes within-block iterations, not every sequential dependency in the report.

Loss & Training¶

The training corpus combines MIMIC-CXR, CheXpert-Plus, ReXGradient, and IU-Xray, with standardized cleaning and normalization into bilingual reports. The authors also mix in a small subset of LLaVA-ReCap-558K to mitigate catastrophic forgetting, but the main text does not provide a fully reproducible mixing ratio. RAD uses the SFT-stage data from continued pretraining and freezes the vision encoder and projector while training the language-model backbone. DCD randomly samples 30,000 examples from the RAD training set proportionally across datasets and discards examples whose teacher trajectories exhibit degenerate repetition loops. The final distillation setup uses forward KL, step-wise weighting, and end-of-sequence cross-entropy; reverse KL appears only as an ablation alternative. The supplied main text does not specify all optimization hyperparameters, the termination-loss coefficient, or the teacher update schedule, so self-distillation should not be interpreted as implying an EMA update. Some earlier mathematical expressions are damaged in the text extraction; this note retains only the clearly readable core objective from Algorithm 1 rather than reconstructing corrupted formulas.

Key Experimental Results¶

Main Results¶

Evaluation samples 2,000 English and 2,000 Chinese reports from each normalized MIMIC-CXR, CheXpert-Plus, and ReXGradient test set; MIMIC-CXR explicitly uses the official patient-level test split. Clinical-content evaluation uses the positive-finding portion of RaTEScore and SemScore, which the authors describe as insensitive to negative findings; neither is clinical diagnostic accuracy. The table below selects clinical-content results from Table 1, page 12; TPF expresses relative tokens generated per forward pass, and TPS is measured in tokens/s. Speed appears as shared model-level columns in the original table and should not be interpreted as a separate measurement specific to each dataset row below.

Dataset	Model	RaTEScore ↑	SemScore ↑	TPF ↑	TPS ↑
CheXpert-Plus	Lingshu-7B	31.34	27.54	1.00	53.70
CheXpert-Plus	ECHO-Base	59.12	52.64	1.89	48.86
CheXpert-Plus	ECHO-blk8	56.85	51.40	8.00	274.21
CheXpert-Plus	ECHO-blk4	57.40	49.57	4.00	129.19
ReXGradient	Lingshu-7B	20.26	34.82	1.00	53.70
ReXGradient	ECHO-Base	63.02	66.68	1.89	48.86
ReXGradient	ECHO-blk8	59.92	64.00	8.00	274.21
ReXGradient	ECHO-blk4	62.18	66.28	4.00	129.19
MIMIC-CXR	Lingshu-7B	32.79	26.94	1.00	53.70
MIMIC-CXR	ECHO-Base	55.85	46.10	1.89	48.86
MIMIC-CXR	ECHO-blk8	52.97	44.83	8.00	274.21
MIMIC-CXR	ECHO-blk4	53.95	45.57	4.00	129.19

Using the measured TPS values, ECHO-blk8 is \(274.21/53.70\approx5.11\) times faster than Lingshu-7B in reported throughput, not 8 times; 8.00 refers to TPF. Although the Table 1 caption mentions average metrics, the headers separate the three datasets; this note follows those headers without introducing a new aggregate. The gaps against external models include domain continued pretraining and normalization effects and cannot be attributed entirely to DCD; distillation variants sharing ECHO-Base better isolate the distillation strategy.

Ablation Study¶

The following results come from Table 2, page 13: SW denotes step-wise weighting, CE denotes <eos> cross-entropy, and RKL replaces forward KL with reverse KL; rows without RKL use forward KL. The selected metrics are CheXpert-Plus ROUGE-L and RaTEScore, MIMIC-CXR CIDEr, and PPL evaluated with Qwen3-1.7B, where lower PPL is better. The complete configuration matches the ECHO-blk4 values in Table 1, although the Table 2 caption does not explicitly specify block length.

Config	CheXpert ROUGE-L ↑	CheXpert RaTEScore ↑	MIMIC CIDEr ↑	PPL ↓
No SW, no CE, no RKL	52.57	54.87	3.63	23.72
SW	52.44	56.30	3.65	21.07
SW + CE	56.14	57.40	4.05	18.83
SW + RKL	52.51	55.20	3.48	20.23
SW + CE + RKL	53.25	57.25	3.67	21.32

Adding termination CE to SW lowers PPL from 21.07 to 18.83 and raises CheXpert-Plus ROUGE-L from 52.44 to 56.14. This comparison supports the importance of reliable termination for overall text quality, but PPL from another language model cannot replace physician assessment of findings. Reverse KL performs worse, which the authors explain through its tendency to concentrate probability mass; the table establishes the performance change, not an independent proof of that mechanism.

To examine how supervision propagates across stages, the following selection from Table 3, page 15, reports CheXpert-Plus results; normalized and unnormalized training are both evaluated on the paper's normalized test set.

Stage	Training Reports	ROUGE-L ↑	RaTEScore ↑	SemScore ↑
I: Continued pretraining	Normalized	56.89	59.18	52.94
I: Continued pretraining	Unnormalized	23.82	45.59	32.44
II: RAD	Normalized	56.90	59.12	52.64
II: RAD	Unnormalized	23.47	39.60	30.46
III: DCD	Normalized	56.14	57.40	49.57
III: DCD	Unnormalized	18.79	42.08	27.53

Key Findings¶

One-step distillation is not lossless: MIMIC-CXR RaTEScore decreases from 55.85 for ECHO-Base to 52.97 for blk8, so speed gains accompany lower clinical-content scores.
Smaller blocks do not improve every metric: CheXpert-Plus SemScore is 49.57 for blk4 versus 51.40 for blk8, so a conservative configuration is not uniformly better.
Normalization yields large lexical gains and also improves positive-finding metrics; because the references are normalized too, raw clinical reporting requires separate evaluation.
Figure 5, page 14, reports quality saturation at about 60 steps, corresponding to 2.2% of the full RAD data, while TPF continues rising from 1.62 to 2.17, distinguishing efficiency convergence from quality convergence.

Highlights & Insights¶

DCD retains the teacher's conditional distribution at decision time rather than only the final hard label. This choice transfers contextual information from the trajectory through the training target instead of merely reducing sampling steps.
RAD exploits long inputs and relatively short responses in chest X-ray reporting by duplicating only the response. Its savings arise from training-sequence layout rather than discarding visual information.
Termination is treated as behavior requiring dedicated supervision rather than an incidental ordinary-token prediction. The ablation suggests that reducing repetition loops can improve fluency and the completeness of clinical descriptions together.
Cache fusion connects algorithmic acceleration to implementation overhead. Reexamining extra calls per block after compressing denoising to one step provides a concrete engineering lesson for other systems.

Limitations & Future Work¶

Multiple supplementary references remain ??, preventing verification of data statistics, preprocessing details, and the caching proof from the supplied text; the project link is retained as provided by the paper and was not checked online.
The student still predicts per position, and stitched trajectory targets do not automatically yield a joint distribution capable of representing arbitrary correlations; eliminating mean-field limitations is a stronger claim than the experiments establish.
The supplied paper does not show prospective clinical deployment, blinded physician review, or comprehensive external-institution generalization, so automatic text metrics cannot establish readiness for independent clinical use.
Normalization may turn omissions into definite negatives; future work should audit rewritten labels and report false positives, false negatives, and uncertainty rather than only aggregate scores.
Hardware, batch size, and complete timing boundaries for the TPS comparison are insufficiently specified in the supplied main text; 5.11 times is a ratio of reported throughput, not a guarantee about end-to-end report latency on arbitrary systems.
The in-text SemScore citation [43] points to a BERT-based radiology-report labeling reference, leaving the correspondence between the metric name and implementation unclear; this note preserves the table label without inventing a definition.

Versus SDAR-VL, original reference [9]: both follow autoregressive-to-block-diffusion adaptation; ECHO's RAD specifically avoids visual-context duplication and consolidates conversion into one SFT stage.
Versus dParallel, CD4LM, d3LLM, and T3D, original references [8], [31], [38], and [61]: these variants are compared using the same ECHO-Base and distillation data; DCD differs by collecting conditional predictions along the teacher's actual confidence-guided trajectory and adding stability supervision.
Versus Fast-dLLM, original reference [52]: historical KV caching already reduces recomputation; ECHO further removes the dedicated cache-update call in one-step block decoding.
Transferable direction: structured medical reports and other long-context, short-response tasks could test response-asymmetric training and trajectory distillation, but terminology combinations, negation, termination, and reliability across block lengths require fresh validation.

Rating¶

Novelty: 4/5. Conditional trajectory distillation, response-asymmetric adaptation, and cache fusion form a targeted combination, but claims about joint distributions need tighter qualification.
Experimental Thoroughness: 3/5. Three datasets, bilingual evaluation, and component ablations are informative, while clinical validation and reproducibility details remain limited.
Writing Quality: 3/5. The main pipeline is understandable, but unresolved supplementary references, a questionable metric citation, and inconsistent presentation weaken precision.
Value: 4/5. The training and inference designs offer useful lessons for high-throughput medical text generation, but remain a research method rather than a clinically validated system.