Skip to content

Low-Level Dataset Distillation for Medical Image Enhancement

Conference: ECCV2026
Paper: ECCV
Code: https://github.com/xufz123/Med_LLDD
Area: Medical Imaging
Keywords: dataset distillation, medical image enhancement, image super-resolution, low-dose CT restoration, gradient matching

TL;DR

The paper proposes the first dataset distillation framework for low-level medical image enhancement: a shared anatomical prior built from one representative subject is personalized per subject by a learnable subject-specific code (SPG), and subject-aware gradient matching aligns training trajectories, so that distilled data at roughly 99% compression approaches full-data training on CT/MRI super-resolution and low-dose CT restoration.

Background & Motivation

Medical image enhancement — super-resolution, denoising and restoration — aims to turn low-quality (LQ) images into high-quality (HQ) ones that support diagnosis and clinical decision-making. Existing methods almost universally rely on large-scale, high-resolution medical datasets to learn complex pixel-level mappings, which brings heavy training and storage costs and hinders deployment at the clinical end. Dataset distillation (DD) is an attractive remedy: it packs the "training behaviour" of a large dataset into a tiny synthetic set, so that a downstream model trained only on the synthetic set performs comparably to one trained on the raw data. Existing DD methods — bi-level performance matching, parameter matching via gradient matching, and distribution matching over features — are mature on classification, but they all share one premise: multiple samples share a single semantic label. This many-to-one mapping is what makes compression possible, because the synthetic data only has to capture the semantics shared inside a class.

Low-level tasks overturn that premise. Super-resolution and restoration are dense image-to-image mappings with a many-to-many relation between samples and targets: every input has its own pixel-level target, and a handful of synthetic images simply cannot constrain such a dense input-output relation. Low-level DD is therefore inherently underdetermined, far harder than its high-level counterpart. The medical setting adds two further complications. First, modalities and degradations are diverse — ×4 downsampling for CT, ×2 downsampling for MRI, and noise added in the projection domain followed by back-projection for low-dose CT — so a single distillation recipe may not transfer. Second, anatomical structure varies substantially across subjects; the paper visualizes gradients from different subjects with t-SNE and finds clearly separated clusters across subjects while gradients within a subject stay close. Forcing the training information of several subjects into one synthetic sample therefore causes conflicts and can collapse training.

The paper's angle follows from that observation: since different subjects share similar anatomical structure, that commonality can be extracted into a shared anatomical prior used as initialization, which constrains the underdetermined problem, and distillation can then be performed per subject, decomposing one large underdetermined problem into a series of smaller, manageable ones. At the same time, distillation is pushed from appearance similarity to training-dynamics similarity: gradient matching makes the optimization trajectory of a network trained on synthetic data align with that of a network trained on the raw data. Core idea: take randomly selected slices of a representative subject as a shared anatomical prior, personalize it into per-subject synthetic HQ data with a learnable subject-specific code, construct LQ–HQ pairs through the task's degradation operator, and align training trajectories by matching gradients separately per subject — downstream users only ever receive the distilled dataset, never raw subject data.

Method

Overall Architecture

The method answers the question of how to distill a dataset for low-level medical enhancement, where there are no class labels and only dense pixel-level mappings. The pipeline runs per subject in parallel. First, a representative subject is picked from the raw data and v of its slices are randomly taken as a shared anatomical prior for initialization. Then, for each subject p, a learnable subject-specific code modulates that prior into synthetic HQ data belonging to that subject (the SPG module). Next, a task-specific degradation operator maps the synthetic HQ data to LQ inputs, which are paired with the raw data's own LQ–HQ pairs and fed to the enhancement network. Finally, the gradients computed on the synthetic pairs and on that subject's raw pairs are compared, and the mismatch drives updates of both the synthetic data and the subject-specific code. Once distillation finishes, only the distilled dataset is delivered downstream; the raw subject data is not used again.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Raw HQ data of multiple subjects"] --> B["Shared anatomical prior init<br/>random slices of one representative subject"]
    B --> C["Subject-aware personalized generation<br/>subject code modulation + fidelity residual"]
    C --> D["Task-specific degradation pairing<br/>D(·) builds LQ–HQ pairs"]
    D --> E["Subject-aware gradient matching<br/>align training trajectories per subject"]
    A -->|gradients of raw LQ–HQ pairs| E
    E -->|update synthetic data and subject code| C
    E --> F["Distilled dataset, M≪N<br/>the only artifact shared downstream"]

Key Designs

1. Shared anatomical prior as initialization: constraining the underdetermined pixel-level problem with a global structural prior

The difficulty of low-level DD comes from having to cover a many-to-many dense mapping with a few synthetic images; under such weak constraints, randomly initialized synthetic data learns almost nothing — in the ablation, swapping the initialization for random noise collapses PSNR to 11.54 dB (NRI=5, IPS=1). The paper exploits the fact that subjects scanned under the same protocol share highly similar anatomy: one representative subject is chosen from the raw dataset and v slices are randomly drawn from its 3D volume (CT or MRI) to form \(U\in\mathbb{R}^{v\times h\times w}\), which serves as the shared initialization for every subject. This does two things at once. It shrinks the solution space from "arbitrary images" to "anatomically plausible images," alleviating the underdetermination; and it supplies a structural basis shared across subjects, so each subject only has to learn a deviation from that basis instead of an entire anatomy from scratch. The authors verify that this initialization is robust to the choice of representative subject: using subject #1 or subject #3, CT super-resolution at IPS=5 yields 34.45 dB and 34.27 dB respectively.

2. Subject-aware personalized generation: rewriting the shared prior into one subject's synthetic data with a subject-specific code

The shared prior only carries what is common, while anatomical differences between subjects are real; the paper explicitly gives up on representing several subjects with one synthetic set and distills per subject instead, which both preserves structural fidelity and decomposes one large underdetermined problem into P smaller ones. The SPG module performs this rewrite with two components: a learnable subject-specific code \(d_p\in\mathbb{R}^{q}\) for subject p (q=2 by default) and a subject-agnostic convolutional layer \(\theta^c\) shared across subjects. Modulation works by duplicating, scaling, and concatenating: the prior \(U\) is copied q times, the k-th copy is scaled by the k-th scalar of the subject code, and the copies are concatenated along the channel dimension into a tensor of shape \(\mathbb{R}^{v\times q\times h\times w}\), so each component of the subject code controls the strength of one anatomical pathway. A shared convolutional layer then fuses these q pathways and reshapes them into the number of images each subject should have, denoted IPS (written \(i\)). Because convolution alone would smooth away pixel-level detail, a pixel-level fidelity preservation step randomly takes one slice \(u_p\in\mathbb{R}^{h\times w}\) from the prior, duplicates it \(i\) times, and adds it back as a residual:

\[\tilde{Y}_p=\mathrm{Conv}\big(f(U,d_p);\theta^c\big)+\mathbf{u}_p\]

Here \(\tilde{Y}_p\in\mathbb{R}^{i\times h\times w}\) is the synthetic HQ data of subject p. The residual is not decoration: removing it (Ours‡) drops PSNR from 32.73 dB to 16.95 dB at IPS=1 and from 34.45 dB to 30.55 dB at IPS=5, making this module the largest single contributor to actually writing anatomy into synthetic pixels. Note that the subject code is learned independently per subject, so what is synthesized is not a copy of the raw subject image but a synthetic sample shaped by training dynamics that still carries that subject's structural characteristics.

3. Task-specific degradation pairing: with no labels, the degradation operator manufactures the supervision

Classification DD draws supervision from shared labels; low-level DD has none, so the paper instead uses the task's own degradation model. The construction is direct: the SPG output is treated as HQ ground truth \(\tilde{y}_m\), and a degradation operator \(D(\cdot)\) is applied to it to obtain the paired LQ input \(D(\tilde{y}_m)\), forming the synthetic paired set \(\mathcal{S}_p^l=\{D(\tilde{y}_m),\tilde{y}_m\}\); the same operator is applied to the raw data to form \(\mathcal{T}_p^l=\{D(y_n^p),y_n^p\}\). \(D(\cdot)\) switches with the task: downsampling for super-resolution (CT images of 512×512 shrunk by ×4; MRI resized to 256×256 and then downsampled by ×2), and noise added to projections followed by back-projection for low-dose CT (LDCT) restoration, with the photon count set to \(10^4\). The point is not just to create data but to encode what the task is into the distillation itself — the distilled set is tied to that task's degradation model, and adapting to a new task only requires swapping \(D(\cdot)\) rather than changing the method.

4. Subject-aware gradient matching: aligning training trajectories per subject instead of sharing one synthetic set

The first three steps make the synthetic data look like medical images, but what low-level DD must really preserve is training behaviour: a network trained on the synthetic set should follow a parameter trajectory close to that of a network trained on the raw data. The t-SNE plot shows gradient distributions clearly separated across subjects and clustered within a subject, so matching is done per subject rather than globally — splitting the dense mapping into sub-mappings and avoiding the cross-subject conflicts that would otherwise collapse training. For subject p, gradients \(\nabla\theta_p\) and \(\nabla\theta'_p\) are computed on the raw paired set \(\mathcal{T}_p^l\) and the synthetic paired set \(\mathcal{S}_p^l\) respectively, and their cosine similarity is turned into a matching loss (smaller means better aligned directions):

\[\mathcal{L}_{GM}=1-\frac{\nabla\theta_p\cdot\nabla\theta'_p}{\lVert\nabla\theta_p\rVert\,\lVert\nabla\theta'_p\rVert}\]

⚠️ This equation is garbled in the cached PDF text; the form above is reconstructed from the standard gradient-matching formulation (one minus cosine similarity) — refer to the original paper. This term is the channel through which subject-specific training knowledge enters the synthetic data: the synthetic pixels need not look like the raw slices, as long as the gradients they produce together with the degradation operator point in the right direction. That is exactly why the authors stress that they do not copy subject anatomy into the distilled data but align the optimization trajectory. The task loss uses MSE to measure the pixel-level difference between the enhanced output and the HQ ground truth:

\[\mathcal{L}^{l}_{MSE}=\big\lVert\phi\big(D(\mathbf{y});\theta\big)-\mathbf{y}\big\rVert^2\]

The synthetic side is computed analogously with \(\mathcal{S}_p^l\) substituted for the paired data. The overall distillation target is a per-subject parameter matching, \(\min\sum_{p=1}^{P}\mathcal{L}_{PM}\big(\theta_{\mathcal{S}_p^l},\theta_{\mathcal{T}_p^l}\big)\), i.e. making the parameters learned from the synthetic set stay close to those learned from that subject's raw set.

A Worked Example

Take CT super-resolution with NRI=10 and IPS=5. A representative subject is picked, and 10 slices are randomly drawn from its 3D CT volume to form the shared prior \(U\in\mathbb{R}^{10\times 512\times 512}\) (NRI=10, corresponding to a 98% compression rate — 98% of the raw images are not involved in the initialization). For subject #1, a learnable code of dimension 2 is taken; \(U\) is duplicated into 2 copies, each scaled by one scalar of the code and concatenated into a \(\mathbb{R}^{10\times 2\times 512\times 512}\) tensor, which a shared convolution compresses into 5 synthetic HQ images of size \(512\times512\). One slice is randomly drawn from \(U\), duplicated 5 times, and added back, giving \(\tilde{Y}_1\in\mathbb{R}^{5\times 512\times 512}\). Each of the 5 images is downsampled ×4 to form LQ inputs, which together with subject #1's raw LQ–HQ pairs produce gradients; the cosine mismatch updates both the 5 synthetic images and the subject code over 2,000 iterations. Running this for all 10 subjects yields 50 synthetic images totalling about 822 KB, against 39.9 MB for the full CT dataset (storage reduction of 97.98%). Downstream users receive only those 50 images, generate LQ inputs by downsampling, and train SRCNN for 300 epochs, reaching PSNR 34.57 dB and SSIM 92.83 on a test set of two held-out subjects; training on the full 39.9 MB dataset gives 36.10 dB / 94.86 as a reference.

Loss & Training

Distillation uses MSE as the task loss on the raw paired set and the synthetic paired set separately, and the cosine gradient mismatch \(\mathcal{L}_{GM}\) to update the synthetic data and the subject code, optimized for 2,000 iterations. All experiments run in PyTorch on a single RTX 3090. SRCNN is used for super-resolution distillation and REDCNN for restoration distillation; the subject code dimension defaults to 2 (the hyper-parameter study shows 2 dimensions is the most stable under both NRI settings). Downstream, CT super-resolution and CT restoration networks are trained for 300 epochs, while MRI super-resolution networks are trained for 600 epochs. For reliability, each experiment generates 3 independent distilled datasets and runs 5 training-testing runs per dataset, reporting mean and variance over 15 test scores; coreset baselines are treated the same way on their selected subsets. Since low-level tasks have no class concept, dataset size is measured by NRI (the number of raw images used for initialization or selection), and the compression rate CR is defined as the ratio of non-selected raw images to the total number of images in the raw dataset.

Key Experimental Results

Main Results

CT uses the public NIH-AAPM-Mayo Low-Dose CT Grand Challenge data (super-resolution: 50 images per subject from 10 subjects; restoration: all images from 10 subjects; two further subjects for testing), and MRI uses Calgary-Campinas-359 (100 high-resolution images per subject from 10 subjects, two for testing). The comparison covers five coreset-selection baselines: Random (images sampled from all subjects), Random* (sampled within a single subject), Uniform (equally spaced sampling), Herding, and K-Center Greedy.

Modality / Task NRI=10 PSNR (dB)↑ SSIM↑
CT ×4 super-resolution (SRCNN) Full Data 36.10±0.02 94.86±0.03
Random 32.66±0.41 91.06±0.51
Random* 32.76±0.19 90.78±0.69
Uniform 32.46±0.28 90.67±0.26
Herding 32.86±0.26 91.18±0.48
K-Center 32.92±0.23 91.34±0.28
Ours IPS=1 32.93±0.28 90.15±1.90
Ours IPS=5 34.57±0.14 92.83±0.28
MRI ×2 super-resolution (SRCNN) Full Data 29.07±0.00 90.46±0.02
Random / Random* / Uniform 26.80 / 26.74 / 26.77 83.63 / 83.54 / 83.79
Herding / K-Center 26.72 / 26.76 83.62 / 83.70
Ours IPS=1 27.02±0.28 77.04±8.22
Ours IPS=5 28.35±0.10 88.27±0.33
CT low-dose restoration (REDCNN) Full Data 28.90±0.01 58.08±0.05
Best baseline (K-Center, NRI=10) 28.11±0.05 56.78±0.32
Ours IPS=1 / IPS=5 (NRI=10) 28.38±0.14 / 28.09±0.03 57.43±0.94 / 56.36±0.42

On CT super-resolution at NRI=10, IPS=5, the method beats the strongest coreset baseline (K-Center) by 1.65 dB PSNR and 1.49 SSIM; on MRI super-resolution it reaches about 97% of full-data training, as stated in the paper. Cross-architecture generalization distills with SRCNN and evaluates on other backbones: on EDSR the method reaches 35.48 dB / 94.57, higher even than its own SRCNN evaluation of 34.79 dB / 93.16, indicating the synthetic data carries transferable training information rather than overfitting to one architecture. The scalability experiment extends CT training to 48 subjects (gradient accumulation is performed every 5 subjects before updating the synthetic samples, since computing gradients over all subjects at once is expensive), and IPS=5 attains 36.08 / 36.07 dB at NRI=5/10, closest to the full-data 36.95 dB.

Ablation Study

CT super-resolution with SRCNN; effect of each component and of the initialization strategy:

Config NRI IPS=1 PSNR / SSIM IPS=5 PSNR / SSIM Storage↓
Full Data — 36.10±0.02 / 94.86±0.03 same 39.9 MB
Ours† random-noise init 5 11.54±0.28 / 45.79±0.36 11.30±0.19 / 45.55±0.23 817 KB (−98.00%) / 4085 KB (−90.00%)
Ours‡ w/o pixel-level fidelity preservation 5 16.95±1.40 / 51.57±1.83 30.55±0.06 / 88.11±0.16 410 KB / 412 KB (−98.99%)
Ours full model 5 32.73±0.22 / 90.50±0.78 34.45±0.20 / 92.07±0.93 410 KB / 412 KB (−98.99%)
Ours full model 10 32.93±0.28 / 90.15±1.90 34.57±0.14 / 92.83±0.28 819 KB (−97.99%) / 822 KB (−97.98%)

Initialization and prior-sampling strategies (CT super-resolution):

Initialization / sampling IPS=1 PSNR / SSIM IPS=5 PSNR / SSIM
Random (noise initialization) 18.41±0.84 / 54.50±1.13 31.84±2.37 / 81.23±8.99
Average (slice-wise mean across aligned subjects) 27.20±8.43 / 73.60±20.49 30.29±0.27 / 86.10±0.53
Representative subject #1 (default) 32.73±0.22 / 90.50±0.78 34.45±0.20 / 92.07±0.93
Representative subject #3 32.26±0.32 / 88.55±2.02 34.27±0.17 / 92.00±0.41
Adjacent sampling 26.24±0.13 / 64.85±0.33 —
Spaced sampling, step=3 / 5 / 10 31.57 / 31.84 / 31.89 —
Random sampling (default) 32.73±0.22 / 90.50±0.78 —

Upper-bound study on IPS: as IPS goes 1 → 5 → 10 → 15, PSNR is 32.73 / 34.45 / 34.56 / 34.77 and SSIM is 90.50 / 92.07 / 92.52 / 92.68 — diminishing returns while storage and computation grow linearly.

Key Findings

  • Pixel-level fidelity preservation is the single most critical component, and its importance grows with IPS. Removing it (Ours‡) leaves only 16.95 dB PSNR at IPS=1, 15.78 dB below the full model, and 30.55 dB at IPS=5, 3.90 dB below. The convolutional output alone lacks reliable high-frequency anatomy, so the prior-slice residual is what welds pixel-level information back in.
  • The shared anatomical prior is irreplaceable, but its source is flexible. Random-noise initialization reaches only 31.84 dB at IPS=5 with a very large variance (±2.37 dB), while the Average prior — aligned slice-wise mean across subjects — is even worse (30.29 dB at IPS=5) because averaging volumes blurs structural boundaries. Swapping the representative subject (#1 vs #3) changes results by only about 0.2 dB, supporting the claim that anatomical priors can be shared across subjects.
  • How the prior is sampled matters more than how many slices are taken. Taking consecutive adjacent slices covers only a local anatomical region and gives just 26.24 dB PSNR / 64.85 SSIM; switching to random sampling across slices raises this to 32.73 dB / 90.50, so coverage rather than density determines prior quality.
  • IPS is an efficiency-performance trade-off. Larger IPS helps but with diminishing returns (1→5 gains 1.72 dB, 10→15 gains only 0.21 dB) while storage grows from 410 KB into the multi-megabyte range.
  • One result runs against the main trend and is flagged honestly. On CT low-dose restoration, IPS=1 (28.38 dB / 57.43) is slightly better than IPS=5 (28.09 dB / 56.36), the opposite of the super-resolution trend. The paper offers no explanation; a plausible guess is that the restoration degradation (projection-domain noise plus back-projection) destroys the differential structure between synthetic images, making extra synthetic samples redundant rather than helpful — but ⚠️ this is speculation, refer to the original paper.

Highlights & Insights

  • Turning dataset distillation from semantic compression into training-trajectory compression. Classification DD is compressible only because of shared labels; the paper argues that the many-to-many mapping of low-level tasks makes the problem underdetermined, and so it stops chasing "synthetic images that look real" and instead aligns the gradient direction produced by synthetic data with that of real data. This moves the applicability boundary of DD from labelled discrete tasks to unlabelled dense regression, a genuinely transferable viewpoint.
  • The "shared basis + per-subject personalization" decomposition is remarkably economical. Cross-subject anatomical commonality is carried by a single set of slices (shared, not growing with the number of subjects), while subject differences are carried by a code of only q=2 learned dimensions, which is tiny yet sufficient to distinguish individuals. It is essentially reframing "how much data represents how many individuals" as "a common basis plus sparse coefficients," an idea directly portable to compressing multi-centre or multi-scanner medical data.
  • The degradation operator \(D(\cdot)\) plays the role of the label. With no labels in low-level tasks, the task's degradation model turns unsupervised synthetic data into supervised LQ–HQ pairs and binds the distillation target to the task. This is a cheap and general interface — porting to a new task requires only swapping \(D(\cdot)\), with method and hyper-parameters essentially unchanged.
  • The privacy story lands where it should. The authors emphasize that raw subject data is never explicitly transmitted to downstream users and that the deliverable is only the synthetic dataset (39.9 MB of subject data compressed to 822 KB). Distillation itself still needs the raw data for gradient computation, but the property that downstream training no longer requires holding raw patient data is genuinely valuable in practice.

Limitations & Future Work

  • Distillation still requires the full raw dataset. Gradient matching needs gradients over each subject's raw LQ–HQ pairs, and with 48 subjects the authors must accumulate gradients over every 5 subjects to keep the cost manageable. What the method minimizes is therefore data sharing downstream, not raw-data use during training, and the privacy benefit should be understood with that boundary.
  • The synthetic dataset grows linearly with the number of subjects. Per-subject distillation means P×IPS images in total, so the compression ratio is diluted when there are very many subjects (in the 48-subject experiment CR is 99.92% at NRI=5, but NRI counts initialization images rather than synthetic images, so the evaluation convention favours the method). Readers should note the difference between the NRI/CR convention and a "total synthetic pixels over total raw pixels" measure.
  • IPS being slightly worse at larger values in restoration (see the last key finding) is left unexplained, suggesting that such hyper-parameters depend on task and degradation type, with no selection criterion offered.
  • Limited evaluation scope. Only CT and MRI, and only super-resolution and low-dose restoration, are covered; downstream backbones are limited to SRCNN / REDCNN / EDSR / SCTSRN; only PSNR and SSIM are reported, with no perceptual metric such as LPIPS and no comparison against other data-efficient methods in the low-dose CT literature (e.g. self-supervised or federated approaches).
  • Improvement directions. (1) Replace the single-subject slice prior with a common basis from a low-rank decomposition over multiple subjects' volumes (or with a generative/diffusion prior), removing the residual dependence on which subject is chosen while keeping representativeness. (2) Swap MSE for a perceptual or adversarial loss so distillation aligns perceptual quality rather than pixel error only, and report LPIPS alongside. (3) Bring NRI and IPS under a unified per-subject pixel budget and study budget allocation, yielding a budget-performance curve. (4) Embed per-subject distillation in a federated workflow — each client distills locally and uploads only synthetic data — to obtain communication compression and privacy gains simultaneously.
  • vs classification dataset distillation (DD / DC / DM / MTT and successors): these methods rely on the many-to-one mapping where multiple samples share a label, compressing class-shared semantics through bi-level optimization, gradient matching, or feature-distribution alignment. This paper argues that low-level tasks are dense many-to-many mappings with an independent pixel target per sample, so low-level DD is inherently underdetermined; the replacement is a shared anatomical prior for global constraints, a degradation operator for supervision, and per-subject rather than global gradient matching. The weakness is that the synthetic data has no standalone semantic value (unlike class-wise synthetic images that humans can read); the strength is being the first to push DD into dense prediction.
  • vs coreset selection (Random / Uniform / Herding / K-Center): these methods pick subsets from the raw data, so their information is bounded by the real samples selected, and they must still touch and store raw patient data, which cannot compete with the synthetic-data route on privacy. At equal NRI the coreset baselines cluster together (CT super-resolution PSNR 32.46–32.92), while this method reaches 34.57 at IPS=5; the gap comes from synthesis rather than selection.
  • vs conventional training paradigms for medical image enhancement: the standard practice is end-to-end training on large high-resolution medical data (the Full Data reference here: 36.10 dB with 39.9 MB). It works well but costs storage and sharing, whereas this paper trades about 1% of the storage for about 96% of the PSNR level (34.57 vs 36.10). Its position is an efficient substitute when data cannot be shared, not a replacement that surpasses full-data training.
  • Insight: the key observation making gradient matching viable on low-level tasks is that synthetic pixels need not resemble real pixels as long as they produce the right gradient direction once consumed by the network. The idea extends to other unlabelled dense prediction tasks (deraining, dehazing, depth estimation) for data compression, provided the task has an explicit degradation or forward operator with which to construct pairs.

Rating

  • Novelty: ⭐⭐⭐⭐⭐ First to bring dataset distillation to low-level medical image enhancement; it pinpoints why the "many-to-one is compressible" premise of high-level DD fails under dense mappings, and answers with a self-consistent recipe of shared prior plus per-subject gradient matching.
  • Experimental Thoroughness: ⭐⭐⭐⭐ Covers two modalities (CT/MRI), two task families (super-resolution, restoration), cross-architecture generalization, and a 48-subject scale-up, with each component ablated; however, the metrics are limited to PSNR/SSIM, perceptual metrics and comparisons against domain-specific data-efficient methods are missing, and the anomalous IPS result in restoration is unexplained.
  • Writing Quality: ⭐⭐⭐⭐ The problem statement (Section 3 writing the high- and low-level DD objectives side by side) is clear and motivation and method are tightly coupled; some equations are corrupted in the typeset text (the gradient-matching form and the DD objective), so readers must reconstruct them.
  • Value: ⭐⭐⭐⭐ Offers a workable path to approach full-data training with about 1% of the storage in settings where medical data cannot be shared, with public code; the linear growth in distilled data caused by the single-subject prior and per-subject distillation limits direct application to very large multi-centre datasets.