Solving Diffusion Inverse Problems with Restart Posterior Sampling¶
Conference: ECCV 2026
Paper: ECCV 2026 official page / project page (RePS)
Area: Image Generation
Keywords: diffusion inverse problems, posterior sampling, restart sampling, plug-and-play priors, image restoration
TL;DR¶
RePS extends "restart sampling" from unconditional diffusion (piecewise ODE integration plus periodic large noise injections that wipe out accumulated approximation error) to posterior inference: each round runs a few measurement-conditioned ODE steps to push the sample down to the clean regime, solving a small MAP sub-problem in \(x_0\) space at every step (so no gradient ever back-propagates through the score network), then re-noises the sample back to an annealed restart level and starts again — delivering the strongest overall results on 5 linear and 3 non-linear inverse problems over FFHQ/ImageNet, ahead of DAPS, DPS, DDRM, DDNM and DiffPIR.
Background & Motivation¶
Inverse problems aim to recover a true signal \(x_0\) from indirect, incomplete or noisy observations \(y = h(x_0) + z\), and they are everywhere in medical imaging, computational photography and neuroscience. Rather than hand-crafting analytic priors (total variation, \(\ell_2\) penalties), recent work uses a pre-trained diffusion model as a strong implicit prior. This paper cares only about the "plug-and-play" branch: the unconditional diffusion model stays frozen, and an inference-time algorithm adapts it to arbitrary inverse problems on the same dataset. Two families exist on this branch, and both have a hard limitation. The first decouples optimization from sampling — it denoises/samples unconditionally into pixel space, applies the measurement constraint in \(x_0\) space (DCDP via gradient descent to a MAP estimate, DAPS via Langevin dynamics to genuinely sample the posterior), then uses the forward diffusion process to push the sample back to a lower noise level. It avoids back-propagation, but the reverse diffusion itself is blind to the constraint, so most of the compute goes into generating a sample that may not satisfy the measurement at all. The second family optimizes likelihood and prior jointly; because the two spaces are connected only through the reverse-diffusion network, these methods almost always back-propagate gradients through that network. DPS "lifts" the measurement operator onto noisy samples \(x_t\) and therefore needs the Jacobian of the score; the variational-inference scheme of Graikos et al. makes a round trip backwards through reverse diffusion and forwards through forward diffusion. The compute is heavy, and the formulation is often restricted to linear measurement operators.
The real obstacle is that iterative sampling needs the likelihood score at a noisy sample, \(\nabla_{x_t}\log p(y|x_t)\), whereas the measurement model only supplies the clean-sample quantity \(\nabla_{x_0}\log p(y|x_0)\); there is no generic closed-form bridge between them. The authors notice, however, that the popular DDIM sampler offers a way out: a single ODE update from \(x_t\) to \(x_{t-1}\) can be written entirely in terms of the posterior mode \(\mathbb{E}[X_0|x_t,y]\), and finding that \(x_0\) — by a few gradient-descent iterations, say — never requires passing through the score network. The reason is plain: this sub-problem asks "what clean image \(x_0\) looks like a denoised version of the current sample and respects the measurement?", not "what noisy sample \(x_t\) is most congruent with this denoiser?", so gradients land only on \(h\).
Approximations still accumulate, though. The authors import a set of findings from unconditional diffusion sampling: with a small NFE budget ODEs win because of their smaller discretization error; with a large budget SDEs overtake them because continuous noise injection contracts accumulated approximation error faster; and "piecewise ODE integration plus periodic large noise injections" — restart sampling — gets the best of both. Xu et al. offered the idea but no simple restart schedule (they re-open at multiple arbitrary noise levels, repeatedly). Core idea: generalize restart sampling from unconditional generation to posterior inference — each round of conditioned ODE integration is the first stage of the restart framework, and the restart is a forward-diffusion re-noising of the just-computed \(x_0\) back to an annealed noise level \(\sigma_r\) (the second stage), which periodically obliterates the accumulated approximation error without ever back-propagating through the score network.
Method¶
Overall Architecture¶
RePS takes a pre-trained unconditional diffusion model \(s_\theta\) (used in the unified VE-SDE form with noise schedule \(\sigma(t)=t\), following Karras et al.), a measurement \(y\) and a differentiable measurement operator \(h\), plus sampling hyper-parameters (\(\sigma_{\max},\sigma_0,\sigma_{\min},\sigma_{\text{res}},N_{\text{res}},N_{\text{ode}},\rho_{\text{ode}},\rho_{\text{res}}\)). The output is a single sample \(x_0\) from the posterior. Nothing is trained; the whole thing is one sampling trajectory built from two alternating stages:
- Stage 1 · conditioned ODE integration: starting from the current noise level \(\sigma_r\), run \(N_{\text{ode}}\) Euler steps of a measurement-conditioned ODE down to \(\sigma_0=0.01\). Each step first obtains the unconditional denoising estimate \(\mathbb{E}[X_0|x_t]\) via Tweedie's formula, then solves a MAP sub-problem in \(x_0\) space that must both respect the measurement and stay close to that estimate, and finally uses the resulting \(\hat x_{0|y}\) to build the ODE update.
- Stage 2 · restart: re-noise the \(x_0\) just produced with the forward diffusion process to the next (lower) restart level \(\sigma_{r-1}\) — that is, resample from \(\mathcal{N}(x_0,\sigma_{r-1}^2 I)\) — and return to stage 1.
The restart levels themselves follow a polynomial schedule annealed from \(\sigma_{\text{restart}}\) down to a fixed \(\sigma_{\min}=0.1\), so the trajectory is a repeated pattern of "one short clean conditioned-ODE segment plus one large noise injection", with the injected noise shrinking each round. The paper stresses that this two-stage view is not unique to RePS: reading the recent posterior-sampling methods along the axes of "how many ODE steps per restart" and "is the constraint inside or after the integration", they all fall inside this framework, and RePS simply occupies a dimension nobody had crossed before.
Key Designs¶
1. Measurement-conditioned ODE steps: replacing the posterior score with a MAP sub-problem in \(x_0\) space
Under the VE-SDE, one Euler step of the posterior-score reverse ODE can be rewritten as a contraction of the current sample toward the conditional posterior mean:
In other words, both the direction and the size of every step are dictated by the posterior mean \(\mathbb{E}[X_0|x_t,y]\) (equivalently, \(\nabla_{x_t}\log p(x_t|y) = (\mathbb{E}[X_0|x_t,y]-x_t)/\sigma^2(t)\)). That mean has no closed form. The authors' substitute is to approximate it by the posterior mode (MAP): from \(p(x_0|x_t,y)\propto p(y|x_0)\,p(x_0|x_t)\) and approximating the unconditional factor \(p(x_0|x_t)\) as \(\mathcal{N}(\mathbb{E}[X_0|x_t],\sigma_t^2 I)\), the MAP sub-problem becomes a two-term least-squares tug-of-war:
The first term demands measurement consistency, the second keeps the estimate from drifting away from what the denoiser finds plausible; in practice \(\sigma_n^2/\sigma_t^2\) is merged into a single tunable, time-invariant weight \(\lambda\). The decisive trade-off is this: Li and Wang solve that optimization in closed form for linear problems, at the price of pinning the sampler to linear (or very special non-linear) operators. RePS gives up the closed form and solves it with a few gradient-descent iterations at every step, so any differentiable \(h\) — linear super-resolution/inpainting/deblurring, or non-linear phase retrieval and HDR — plugs straight in. Because the optimization variable is \(x_0\) rather than \(x_t\), gradients never traverse the diffusion network, so the compute is actually lower than for score-Jacobian methods such as DPS.
2. A simplified restart schedule triggered only in the clean regime
Integrating the conditioned ODE all the way down accumulates error: the score network is imperfect and Euler discretization leaves a residual, and near \(\sigma_0\) that error is easily amplified into visible artifacts. DAPS/DCDP respond by slicing the trajectory into many small segments and applying a constraint plus forward re-noising at the end of each; the restart scheme of Xu et al. re-opens at multiple arbitrary noise levels and repeats each several times, which is parameter-heavy and fiddly to implement. RePS reduces the trigger criterion to a single rule: restart only when the sample has been integrated all the way to \(\sigma_0\) (the noise-free regime), re-noising \(x_0\) with the VE forward process to the restart level, i.e. \(x_r \sim \mathcal{N}(x_0,\sigma_r^2 I)\). The restart-level sequence \(\{\sigma_r\}\) reuses the very same polynomial interpolation used by the ODE simulator (\(\rho_{\text{res}}=15\)) and anneals monotonically from a task-specific \(\sigma_{\text{restart}}\) down to a fixed \(\sigma_{\min}=0.1\). The authors find that concentrating restarts at low noise levels works better, so only \(\sigma_{\text{restart}}\) is tuned while \(\sigma_{\min}\) is shared by every task.
The benefit comes from doing the noise injection in one big dose: a single large re-noising scatters the approximation error accumulated over the preceding steps (the paper's phrase is that a stochastic restart obliterates accumulated approximation error), and because the restart distribution is centred on the very \(x_0\) just computed, the good mode already found is not thrown away — a fundamental difference from re-sampling from scratch. SDE sampling, by contrast, sprinkles a little noise at every step; it also contracts error, but spends a large share of its NFE budget on noise injection. RePS's "several clean ODE steps plus an occasional large noise injection" therefore captures both the low discretization error of ODEs and the error-contraction of SDEs.
3. Multiple conditioned ODE steps per restart: occupying an unvisited dimension of the design space
The authors re-locate recent methods inside a two-dimensional design space: one axis is how many (unconditioned) ODE steps are taken per restart, the other is whether the constraint sits inside the ODE integration or is applied separately afterwards. Near the origin sit Li and Wang and DiffPIR: a single conditioned ODE step that does everything at once (Tweedie denoising plus one conditioning step), which puts them at the intersection of the two axes and makes them essentially equivalent to a "Tweedie special case" of DCDP — the only thing that distinguishes DiffPIR is that it interpolates the MAP estimate back toward the noisy sample before restarting. Along the "integrate freely first, constrain afterwards" axis sit DCDP and DAPS: they run several unconditioned ODE steps and then apply a separate constraint in \(x_0\) space (DCDP by gradient descent to a MAP estimate, DAPS by Langevin dynamics for genuine posterior sampling). The dimension "constraint inside the integration, several steps at a time" was essentially unexplored, and that is exactly where RePS stands: it runs \(N_{\text{ode}}\) conditioned ODE steps per restart (10 in all main experiments).
Why this helps, mechanically: the constraint is compiled directly into the ODE trajectory, so every step steers toward the desired mode; in the alternative, once free sampling has already drifted onto a bad mode, a post-hoc correction struggles to pull it back. Hence faster convergence and better reconstructions. The step-count sensitivity study (super-resolution, total budget fixed at 1000 NFE) supports this: adding ODE steps between restarts helps at first but has a ceiling — LPIPS starts to worsen beyond roughly 20 steps, while PSNR peaks at around 2 steps, revealing a trade-off between "error erased by restarts" and "guidance accuracy within one segment". ⚠️ Note that the main experiments uniformly use 10 steps per restart, which does not line up exactly with the peak of that sensitivity curve; consult the supplementary hyper-parameter table for the actual settings.
A Worked Example¶
Take 4× super-resolution on FFHQ. Here \(h\) is 4× bicubic downsampling and \(\lambda\) has been tuned for the task. Sampling starts at \(\sigma_{\max}=100\) with \(x_K\sim\mathcal{N}(0,100^2 I)\). In the first round, 10 conditioned ODE steps walk from \(\sigma=100\) down to \(\sigma_0=0.01\) along the polynomial schedule with \(\rho_{\text{ode}}=7\): each step first asks the model for \(\mathbb{E}[X_0|x_t]\), then runs a few gradient-descent iterations on \(x_0\) to pull the reconstruction toward "downsampled equals \(y\)", yielding \(\hat x_{0|y}\); the contraction formula above then advances the sample from \(x_t\) to \(x_{t-\Delta t}\). After 10 steps we hold an approximately clean reconstruction. The restart then re-noises it to \(\sigma_{\text{restart}}\) via \(x_r\sim\mathcal{N}(x_0,\sigma_r^2 I)\) (that level is tuned per task and reported in the supplementary material) and another 10 conditioned ODE steps run. The cycle repeats, each round's restart level annealing downward along \(\rho_{\text{res}}=15\) until \(\sigma_{\min}=0.1\); the final \(x_0\) is returned. The budget is about 1000 NFEs for linear tasks and 4000 for non-linear ones. This makes plain what "restart" actually restarts: it does not discard the result and begin from scratch — it re-scatters the current, already decent image up to a higher noise level so the next ODE segment starts from a cleaner error state that is still near the solution.
Loss & Training¶
RePS is entirely training-free: there is no module to train and no loss function, only inference-time sampling hyper-parameters. Globally fixed: \(\sigma_{\max}=100\), \(\sigma_0=0.01\), \(\sigma_{\min}=0.1\), 10 conditioned ODE steps per restart, \(\rho_{\text{ode}}=7\), \(\rho_{\text{restart}}=15\). Tuned per task: the starting restart level \(\sigma_{\text{restart}}\), the number of gradient updates \(N\) per MAP sub-problem, the learning rate \(\eta\) and the weight \(\lambda\) (values in the supplementary material). Both pre-trained checkpoints are publicly released models: Dhariwal and Nichol's for ImageNet 256×256, and Chung et al.'s (DPS) for FFHQ 256×256. Although both were trained in the DDPM/VP-SDE framework, the unified VE-SDE sampler is used to drive them, following Karras et al.
Key Experimental Results¶
Main Results¶
Evaluation uses 100 validation images each from ImageNet 256×256 and FFHQ 256×256. The five linear tasks are: 4× super-resolution (bicubic downsampling), box inpainting (a randomly placed 128×128 box), random inpainting (70% of pixels masked), Gaussian deblurring (61×61 kernel, standard deviation 3.0) and motion deblurring (61×61 kernel, standard deviation 0.5). The three non-linear tasks are: phase retrieval (Fourier magnitude only, 2× oversampling, four reconstructions per measurement with the best reported), non-linear deblurring (using the blur-kernel network released by Tran et al.) and HDR reconstruction (pixel intensities compressed by a factor of 2). Gaussian noise with \(\sigma_n=0.05\) is added to the measurements in every task. Baselines are Li and Wang, DAPS, DDRM, DDNM, DPS, DCDP, FPS-SMC and DiffPIR (linear), plus DAPS, DPS, RED-diff and DCDP (non-linear); metrics are PSNR↑ / SSIM↑ / LPIPS↓.
Table 1: Linear inverse problems on FFHQ (PSNR / SSIM / LPIPS)
| Method | 4× super-resolution | Box inpainting | Random inpainting | Gaussian deblurring | Motion deblurring |
|---|---|---|---|---|---|
| RePS (ours) | 29.95 / 0.845 / 0.155 | 24.32 / 0.844 / 0.122 | 32.49 / 0.899 / 0.090 | 29.92 / 0.841 / 0.146 | 32.27 / 0.875 / 0.115 |
| Li and Wang | 29.21 / 0.805 / 0.256 | 24.75 / 0.817 / 0.167 | 34.11 / 0.926 / 0.071 | 29.89 / 0.831 / 0.217 | 29.90 / 0.766 / 0.207 |
| DAPS (official code) | 29.55 / 0.793 / 0.186 | 24.88 / 0.755 / 0.174 | 30.94 / 0.807 / 0.155 | 29.75 / 0.794 / 0.177 | 31.80 / 0.843 / 0.135 |
| DPS | 25.86 / 0.753 / 0.269 | 22.51 / 0.792 / 0.209 | 25.46 / 0.823 / 0.203 | 25.87 / 0.764 / 0.219 | 24.52 / 0.801 / 0.246 |
| DDRM | 26.58 / 0.782 / 0.282 | 22.26 / 0.801 / 0.207 | — | 24.93 / 0.732 / 0.239 | — |
| DDNM | 28.03 / 0.795 / 0.197 | 24.47 / 0.837 / 0.235 | 29.91 / 0.817 / 0.121 | 28.20 / 0.804 / 0.216 | — |
| DCDP | 28.66 / 0.807 / 0.178 | 23.89 / 0.760 / 0.163 | 30.69 / 0.842 / 0.142 | 27.50 / 0.699 / 0.304 | 25.08 / 0.512 / 0.364 |
| FPS-SMC | 28.42 / 0.813 / 0.204 | 24.86 / 0.823 / 0.146 | 28.21 / 0.823 / 0.261 | 26.54 / 0.773 / 0.253 | 27.39 / 0.826 / 0.227 |
| DiffPIR | 26.64 / — / 0.260 | — | — | 27.36 / — / 0.236 | 26.57 / — / 0.255 |
Table 2: Linear inverse problems on ImageNet (PSNR / SSIM / LPIPS)
| Method | 4× super-resolution | Box inpainting | Random inpainting | Gaussian deblurring | Motion deblurring |
|---|---|---|---|---|---|
| RePS (ours) | 26.12 / 0.708 / 0.259 | 20.71 / 0.787 / 0.186 | 28.97 / 0.829 / 0.115 | 26.21 / 0.703 / 0.255 | 28.95 / 0.801 / 0.169 |
| Li and Wang | 25.68 / 0.682 / 0.307 | 21.39 / 0.779 / 0.217 | 29.67 / 0.861 / 0.114 | 25.75 / 0.681 / 0.296 | 27.70 / 0.733 / 0.238 |
| DAPS (official code) | 25.70 / 0.654 / 0.285 | 21.40 / 0.721 / 0.228 | 27.72 / 0.742 / 0.177 | 26.00 / 0.668 / 0.260 | 28.87 / 0.781 / 0.171 |
| DPS | 21.13 / 0.489 / 0.361 | 18.94 / 0.722 / 0.257 | 23.52 / 0.745 / 0.297 | 20.31 / 0.598 / 0.397 | 18.96 / 0.629 / 0.423 |
| DDRM | 22.62 / 0.521 / 0.324 | 18.63 / 0.733 / 0.254 | — | 21.26 / 0.564 / 0.443 | — |
| DDNM | 23.96 / 0.604 / 0.475 | 21.64 / 0.748 / 0.319 | 31.16 / 0.841 / 0.191 | 28.06 / 0.703 / 0.278 | — |
| FPS-SMC | 24.82 / 0.703 / 0.313 | 22.16 / 0.726 / 0.208 | 24.52 / 0.701 / 0.316 | 23.91 / 0.601 / 0.387 | 24.52 / 0.647 / 0.326 |
| DiffPIR | 23.18 / — / 0.371 | — | — | 22.80 / — / 0.355 | 24.01 / — / 0.366 |
Table 3: Non-linear inverse problems (FFHQ / ImageNet, mean ± std)
| Task | Method | FFHQ PSNR | FFHQ SSIM | FFHQ LPIPS | ImageNet PSNR | ImageNet SSIM | ImageNet LPIPS |
|---|---|---|---|---|---|---|---|
| Phase retrieval | RePS | 30.41±4.45 | 0.824±0.119 | 0.152±0.107 | 20.12±7.18 | 0.449±0.260 | 0.419±0.184 |
| Phase retrieval | DAPS | 30.72±3.15 | 0.809±0.081 | 0.157±0.073 | 22.32±6.51 | 0.514±0.219 | 0.343±0.155 |
| Phase retrieval | DPS | 17.64±2.97 | 0.441±0.129 | 0.410±0.090 | 16.81±3.61 | 0.427±0.143 | 0.447±0.099 |
| Phase retrieval | RED-diff | 15.60±4.48 | 0.398±0.195 | 0.596±0.092 | 14.98±3.75 | 0.386±0.057 | 0.536±0.129 |
| Phase retrieval | DCDP | 28.65±8.09 | 0.781±0.217 | 0.203±0.196 | — | — | — |
| Non-linear deblurring | RePS | 29.02±1.76 | 0.797±0.031 | 0.165±0.030 | 27.58±3.28 | 0.745±0.082 | 0.191±0.056 |
| Non-linear deblurring | DAPS | 28.79±1.54 | 0.781±0.033 | 0.177±0.030 | 27.47±3.21 | 0.737±0.084 | 0.198±0.052 |
| Non-linear deblurring | DPS | 23.39±2.01 | 0.623±0.082 | 0.278±0.060 | 22.49±3.20 | 0.591±0.101 | 0.306±0.081 |
| Non-linear deblurring | RED-diff | 30.86±0.51 | 0.795±0.028 | 0.160±0.034 | 30.07±1.41 | 0.754±0.023 | 0.211±0.083 |
| Non-linear deblurring | DCDP | 27.92±2.64 | 0.779±0.067 | 0.183±0.051 | — | — | — |
| HDR reconstruction | RePS | 27.96±3.54 | 0.872±0.080 | 0.145±0.071 | 26.37±4.05 | 0.843±0.117 | 0.157±0.103 |
| HDR reconstruction | DAPS | 27.52±3.58 | 0.840±0.084 | 0.157±0.064 | 26.50±4.70 | 0.812±0.140 | 0.170±0.113 |
| HDR reconstruction | DPS | 22.73±6.07 | 0.591±0.141 | 0.264±0.156 | 19.23±2.52 | 0.582±0.082 | 0.503±0.106 |
| HDR reconstruction | RED-diff | 22.16±3.41 | 0.512±0.083 | 0.258±0.089 | 22.03±5.90 | 0.601±0.094 | 0.274±0.198 |
Ablation Study¶
| Axis | Config | Result | Note |
|---|---|---|---|
| Sampler type (FFHQ Gaussian deblurring) | Same conditioned ODE, no restart | Converges fastest at low NFE thanks to small discretization error; overtaken at high NFE | Plain ODE sampler baseline |
| Sampler type | Euler–Maruyama SDE with identical conditioning and continuous noise injection | Better than the plain ODE at high NFE; stochastic noise contracts accumulated error | Equivalent to sprinkling noise every step |
| Sampler type | RePS | Best convergence speed and best PSNR / LPIPS across the whole NFE range | Short ODE segments plus periodic large noise injections |
| Conditioned ODE steps between restarts (super-resolution, 1000 NFE fixed) | 1 step (≈ Li and Wang / DiffPIR) | The leftmost point is the single-step conditioned-ODE limit | Sits at the origin of the design space |
| Conditioned ODE steps between restarts | ~2 steps | PSNR peaks here | Trade-off between guidance accuracy and restart frequency |
| Conditioned ODE steps between restarts | 10 steps (main-experiment default) | Best overall behaviour | Consistent with the supplementary hyper-parameter table |
| Conditioned ODE steps between restarts | Beyond ~20 steps | LPIPS starts to worsen | One ODE segment too long; error re-accumulates |
| Pixel vs latent (FFHQ, supplementary) | RePS (latent) | Slightly worse than the pixel-space version | The method extends readily to latent diffusion |
| Posterior calibration (box inpainting / phase retrieval, supplementary) | ensemble skill / spread / spread-to-skill | Non-trivial ratios; no collapse to a point estimate | The spatial spread map for box inpainting shows more uncertainty inside the mask than outside |
| Sampling time (max NFE per task, 1k linear / 4k non-linear) | RePS vs DAPS, per-image timing | Slightly faster on linear tasks, slightly slower on non-linear ones | Overall comparable; differences are small |
(The sampler-type and ODE-step sensitivities above are given in the original as curves in Fig. 3 with no standalone numeric table; this table paraphrases their conclusions — read exact coordinates off the paper.)
Key Findings¶
- Best overall, but not best in every cell. RePS has the broadest coverage, yet individual baselines beat it on individual cells: on FFHQ random inpainting Li and Wang wins outright with 34.11 / 0.926 / 0.071 against RePS's 32.49 / 0.899 / 0.090; on ImageNet Gaussian deblurring DDNM reaches PSNR 28.06 versus RePS's 26.21 (though with a worse LPIPS of 0.278); and on ImageNet box inpainting both FPS-SMC (22.16) and DDNM (21.64) beat RePS's 20.71 in PSNR. The paper's claim is therefore "best overall", which per task depends on which metric matters.
- DAPS is the strongest baseline and gets its own discussion. On FFHQ, RePS beats DAPS on every task; on ImageNet it wins four, is competitive on three and loses one — phase retrieval (20.12 vs 22.32 PSNR) is the single substantial loss the authors admit.
- Non-linear deblurring shows a metric split. RED-diff has higher PSNR (FFHQ 30.86 vs 29.02; ImageNet 30.07 vs 27.58) but worse LPIPS (ImageNet 0.211 vs 0.191), so pixel fidelity and perceptual quality are each claimed by a different method on that task.
- Restarts really do combine the strengths of ODE and SDE sampling. The plain ODE converges faster at low NFE and the SDE is stronger at high NFE, while RePS dominates across the entire NFE range — a direct confirmation of the "small discretization error plus periodic error contraction" motivation. The NFE sweep against DAPS on motion deblurring shows the same pattern.
- There is an optimal restart frequency. With the budget fixed at 1000 NFE, too few conditioned ODE steps between restarts (1 step) degenerates into Li and Wang / DiffPIR, while too many (>20) let error re-accumulate inside a single ODE segment and worsen LPIPS; PSNR peaks at around 2 steps. ⚠️ The main experiments use 10 steps, which does not match the peak of that curve exactly — for deployment, follow the supplementary hyper-parameter table.
- It really produces posterior samples, not point estimates. On inherently multi-modal tasks (box inpainting and phase retrieval) the ensemble spread and the spread-to-skill ratio are non-trivial, and the spatial spread map for box inpainting shows more uncertainty inside the mask than outside — a meaningful property for scientific applications that need uncertainty quantification.
- Cost is on par with DAPS. Avoiding back-propagation through the score network buys generality (any differentiable operator) rather than raw speed: timed at each task's maximum NFE, RePS is slightly faster on linear tasks and slightly slower on non-linear ones, with small gaps.
Highlights & Insights¶
- The unifying view is itself a contribution. Folding DPS, DDRM, DDNM, DCDP, DAPS, DiffPIR and Li and Wang — methods that look like separate lineages — into a single two-stage framework ("condition/denoise stage + forward-diffusion restart stage") and plotting them on a "ODE steps per restart × is the constraint inside the integration" plane exposes, for instance, that DiffPIR and Li and Wang are equivalent to a DCDP-Tweedie variant, and shows that the dimension RePS occupies was previously empty. This "re-locate existing work on coordinate axes" move transfers to any alternating sampling/optimization algorithm.
- Put the constraint inside the ODE rather than correcting afterwards. Free sampling followed by a post-hoc constraint cannot recover once the trajectory has drifted onto a bad mode, whereas writing the measurement constraint into every update keeps a hand on the wheel throughout. The idea transfers to other "sample plus constraint" settings, such as physically constrained generation and controllable image editing.
- Choosing \(x_0\) as the optimization variable buys "no back-propagation through the score network while still supporting non-linear operators". It is a pragmatic engineering trade: closed forms only exist for linear operators, so giving them up in favour of a few gradient-descent iterations widens the scope from linear to any differentiable \(h\) — and since the variable is not \(x_t\), gradients skip the diffusion network and the whole score-Jacobian cost disappears.
- The restart schedule is pared down to a single hyper-parameter. Xu et al.'s restart re-opens at many noise levels repeatedly; RePS keeps only the rule "restart once the sample reaches \(\sigma_0\), with the restart level annealing", fixes \(\sigma_{\min}\) across tasks, and tunes only \(\sigma_{\text{restart}}\). That makes it nearly a drop-in replacement for an SDE sampler.
- Re-noising the current \(x_0\) rather than re-sampling from scratch is the most easily misread part of the word "restart": the mean preserves the solution just found while the variance erases the error, which amounts to a controlled re-exploration around the solution.
Limitations & Future Work¶
- Clearly behind DAPS on ImageNet phase retrieval (20.12 vs 22.32 PSNR, SSIM 0.449 vs 0.514) — the one substantial loss the authors acknowledge. RePS's variance there is also large (30.41±4.45 PSNR on FFHQ), so its stability is questionable too; and the "generate four reconstructions per measurement, report the best" protocol systematically favours high-variance methods that occasionally get lucky, which makes the conclusion on this task hard to read.
- Limited evaluation scale and inconsistent baseline provenance. Evaluation covers only FFHQ and ImageNet at 256×256 with 100 validation images each, and most baseline numbers are quoted from the DAPS paper (only DAPS was re-run from official code); Li and Wang released no code and were re-implemented by the authors (1000 NFE with gradient descent rather than the closed-form solution). Numbers across papers, implementations and NFE budgets are not directly comparable. ⚠️ Refer to the supplementary material.
- The "task-agnostic" restart schedule still needs per-task tuning. Four quantities — \(\sigma_{\text{restart}}\), the number of gradient updates \(N\) per step, the learning rate \(\eta\) and the weight \(\lambda\) — must be tuned per task. The form is simple, but it is not truly tuning-free, and the main text gives no recommended default table.
- No sensitivity analysis of the restart count itself. Only the number of ODE steps between restarts is swept, yet "how often to restart" is the defining hyper-parameter of a restart method; how to split a fixed NFE budget between "long ODE segments" and "many restarts" is left unanswered.
- Concrete improvement directions: make \(\sigma_{\text{restart}}\) or the restart trigger adaptive — for example, restart when the measurement residual \(\lVert y-h(x_0)\rVert\) stops decreasing — which could reduce the reliance on "best of several reconstructions" on highly ill-posed tasks such as phase retrieval; and couple the number of gradient updates in the MAP sub-problem to the current noise level (few updates at high noise, more at low noise) to save further NFEs.
Related Work & Insights¶
- vs DPS: DPS uses a point estimate to "lift" the measurement operator onto noisy samples \(x_t\), requiring the score Jacobian and back-propagation through it; RePS moves the optimization into \(x_0\) space to solve a MAP sub-problem, so gradients pass only through the measurement operator — no back-propagation through the score network, and non-linear operators are supported natively. In Tables 1/2 DPS is the weakest method throughout (only 24.52 PSNR on FFHQ motion deblurring).
- vs Li and Wang / DiffPIR: both are limiting cases of the RePS framework with a single conditioned ODE step per restart, sitting at the origin of the two-dimensional design space and equivalent to a Tweedie variant of DCDP; the only thing separating DiffPIR is that it interpolates the MAP estimate back toward the noisy sample before restarting. Li and Wang's closed-form solution restricts them to linear problems, while RePS trades that for generality via gradient descent — at the cost of being overtaken on random inpainting, a task Li and Wang handles especially well.
- vs DAPS / DCDP: both run along the "several unconditioned ODE steps, then a separate constraint" axis (DCDP by gradient descent to a MAP estimate, DAPS by Langevin dynamics for genuine posterior sampling). They share RePS's decouple-and-restart skeleton; the difference is whether the constraint sits after the integration or inside it. DAPS is the strongest baseline: RePS beats it on all FFHQ tasks and on four of eight ImageNet tasks, ties on three and loses one.
- vs Xu et al.'s restart sampling: that work is the inspiration, but it applies to unconditional generation and offers an idea rather than a simple schedule (multiple noise levels, each repeated several times). RePS's contribution is to carry restarts over to posterior inference and compress them into a single rule: trigger only at \(\sigma_0\), anneal the restart level.
- vs DDNM / DDRM: null-space-projection and singular-value-decomposition methods that depend strictly on linear operators and usually on a known noise level. They remain strong on structurally simple linear tasks — DDNM reaches 31.16 on ImageNet random inpainting and 28.06 on ImageNet Gaussian deblurring — showing that the classical "no retraining, only projection" route has not been superseded there.
Rating¶
- Novelty: ⭐⭐⭐⭐ A valuable re-framing that carries restart sampling to posterior inference and unifies existing methods in a design space, though both core ingredients (the conditioned ODE and the MAP sub-problem) come from prior work
- Experimental Thoroughness: ⭐⭐⭐⭐ Broad coverage with 5 linear + 3 non-linear tasks on 2 datasets plus calibration and timing analyses; but most baseline numbers are quoted from the DAPS paper and evaluation uses only 100 images
- Writing Quality: ⭐⭐⭐⭐ The design-space discussion maps cleanly onto Algorithm 1 and the motivation chain is well told; some formulas are garbled in the cached layout (e.g. the polynomial schedule definition) and need checking against the paper
- Value: ⭐⭐⭐⭐ Training-free, no back-propagation through the score network, any differentiable operator, and a nearly tuning-free restart schedule — high practical value