Zero-Shot Inference-Time Rectification for Real-World Arbitrary-Scale Super-Resolution

Conference: ECCV 2026
Paper: ECCV Open Access
Area: Image Restoration
Keywords: arbitrary-scale super-resolution, real-world degradation, inference-time rectification, conditional diffusion, contrastive learning

TL;DR

The Continuous Degradation Rectifier (CDR) combines content purification, degradation-scale decoupling, and conditional diffusion to convert real-world low-resolution inputs into the bicubic-style images a frozen super-resolution model already masters; it brings CiaoSR a 4.6858 dB PSNR gain on RealSR ×2, though not all datasets and metrics improve in step.

Background & Motivation

Arbitrary-scale super-resolution (ASSR) models such as LIIF, CiaoSR, and HIIF cast reconstruction as continuous coordinate queries and can handle fractional upsampling factors, yet the pre-trained versions used in this paper were built almost entirely for ideal bicubic downsampling. Real cameras add blur, noise, and compression, and the degradation shifts as the zoom factor changes; supporting a continuous range of output scales does not mean the model has learned to handle a continuously varying distribution of real-world inputs.

Real paired datasets are bounded by the scales at which scenes can be photographed and by hardware conditions. Another line of work first generates degraded LR images and then retrains the SR network on the synthesized pairs — RealDGen, the fixed-scale scheme compared in this paper, is a representative. Carried over scale by scale, that recipe would inflate training and data-preparation costs for arbitrary-scale deployment. This paper instead keeps the existing SR model and moves domain adaptation to the input side.

Core idea: train a rectification proxy that adapts to the target zoom factor, converting real-world inputs into the LR distribution a synthetically pre-trained model already knows — without retraining the SR backbone at test time.

Method

Overall Architecture

The inputs are a real-world LR image and the target zoom factor. Scale-invariant content purification extracts content from the real image; contrastive degradation-scale decoupling extracts degradation and scale information from a scale-matched reference; bandwidth-modulated diffusion guidance fuses these representations so a conditional DDPM emits a canonical bicubic-style LR image, which is then handed to a frozen ASSR backbone for reconstruction.

The reference comes from an HR image unrelated to the input content, bicubically downsampled by the target factor. It supplies cues about the target degradation domain and the scale — it is not a paired ground truth. Training optimizes the content, degradation, and scale encoders in sequence with freezing between stages, and then trains the diffusion model; this is the authors' design for keeping the conditioning signals from interfering with one another, not a mathematical proof that the representations are strictly independent.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Real-world LR image"] --> B["Scale-invariant content purification"]
    R["Unpaired HR reference<br/>downsampled at the target scale"] --> C["Contrastive degradation-scale decoupling"]
    B --> D["Bandwidth-modulated diffusion guidance"]
    C --> D
    D --> E["Canonical bicubic LR image"]
    E --> F["Frozen SR backbone<br/>reconstruction at the target scale"]

Key Designs

1. Scale-invariant content purification: make the discrete bottleneck reconstruct clean targets instead of copying input degradations

During training, high-order synthetic degradations are applied to HR images, which are then downsampled at continuous factors within [1, 4]. A VQ-VAE encoder embeds the degraded image and quantizes the latents to a discrete codebook via nearest-neighbor lookup; the decoder's target is not the degraded input itself but the clean bicubic version of the same HR image at the same factor. Supervision is thus spatially aligned while requiring the model to suppress the synthetic degradations.

The authors interpret the finite codebook as a scale-normalizing anchor: supervision across zoom factors encourages the model to retain content and weaken degradation-driven variation. Note that a finite codebook does not automatically make high-frequency artifacts un-encodable, let alone guarantee that any real-world image is fully purified. This is a supervised representation bottleneck plus the authors' mechanistic reading of it. Once the content encoder converges it is frozen, so later gradients cannot alter the learned representation.

2. Contrastive degradation-scale decoupling: cross positive/negative samples push the model to attend to scale rather than content

In the first step, the content encoder is frozen and the degradation encoder with its decoder are trained at a preset baseline factor — ×2 in the paper. Fixing the factor controls the training variable and keeps scale variation out of the degradation representation; it is not a logical proof that the encoder has become invariant to all factors. Once the degradation encoder converges it is frozen, and the scale encoder is trained next.

The key to scale training is the triplet: the anchor is image A's clean downsample at scale s, the positive is another image B downsampled at the same scale s, while the negative is image A itself downsampled at a different factor. The positive changes content but not scale, the negative changes scale but not content; a triplet margin loss encourages the representation to cluster by scale rather than by image semantics. The scale encoder also receives the zoom factor explicitly, injected via LIIF's cell representation into the positional bias of SwinIR blocks, so scale recognition cannot be described as a fully blind estimate from the image alone.

In addition, a scale-adaptive decoder fuses the anchor image's content, degradation, and scale representations and reconstructs the anchor image with an L2 loss, limiting how much useful information the decoupling process discards. The authors call this "orthogonal decoupling", but the cached full text offers no proof of statistical independence or completeness; it should be read as a training objective, not a guaranteed property. The triplet formula is corrupted in the cache, so this note keeps the verifiable positive/negative construction and the loss type rather than inventing the exact norm and reduction.

3. Bandwidth-modulated diffusion guidance: let the scale reshape the conditional kernel instead of acting as an appended value

The degradation and scale features are first concatenated and passed through a two-layer linear network that produces a channel-wise 3×3 base kernel. The scale feature separately goes through a three-layer linear network that predicts a per-channel scaling and bias to modulate the base kernel; the modulated kernel then depth-wise convolves the purified content, forming the DDPM condition. The crucial point: the zoom factor not only enters the encoder but participates in generating the conditional filter itself.

Following the paper's description of the channel-wise affine transform, the core mechanism can be summarized as:

\[ \hat K=\alpha\odot K_{\mathrm{base}}+\beta. \]

where the scaling and bias are predicted per channel and broadcast across the spatial kernel. The original equation is corrupted in the cached text; this expression restates the textual mechanism. The authors read it as a simulation of optical blur bandwidth, but a channel-wise affine transform is not a calibrated physical optics model and does not by itself guarantee any bandwidth-stretching relation.

Finally, with all encoders frozen, a conditional DDPM is trained to map complex synthetically degraded inputs to the clean bicubic distribution. At test time only the content source is swapped for the real-world LR image; degradation and scale cues come from the scale-matched reference, and the output goes to the frozen ASSR. The ×5 and ×6 extrapolation is supported by experiments, but effectiveness at arbitrary factors does not follow from the architecture alone.

A Worked Example

For a target of ×3.5, the real photo first passes through the content encoder and the quantization bottleneck to form the content representation. A different, unrelated HR image is downsampled ×3.5 to provide the canonical reference; the degradation encoder reads the reference, and the scale encoder reads both the reference and the zoom factor.

The base-kernel and affine networks then generate the conditional kernel, injecting the reference-side information into the real image's content, after which the DDPM synthesizes a bicubic-style LR image. A frozen LIIF or other tested backbone queries the output at ×3.5. This example illustrates the data flow — it does not mean the paper reports the full sampling trajectory for this image; content recovery can still carry errors, so the result cannot be called physically lossless artifact removal.

Loss & Training

The content module uses the standard VQ-VAE objective; after fixed-factor degradation reconstruction the degradation encoder is frozen; the scale module combines the triplet loss with L2 reconstruction; finally the conditional DDPM is trained. The decoupling modules are trained on DIV2K, the diffusion model on Flickr2K, high-order degradations follow Real-ESRGAN, and the content and diffusion stages cover continuous factors within [1, 4].

Implementation uses PyTorch on an RTX 3090. Inference references are produced by downsampling random DF2K images at the target factor. "Zero-shot" here means no real-world paired training and no fine-tuning of the backbone on the target domain — not that CDR was never trained, and it differs from ZSSR's self-supervised optimization on a single test image. The cache lacks sufficient information to support a precise inference-cost assessment.

Key Experimental Results

Main Results

PSNR and SSIM are higher-better; LPIPS is lower-better. The table below excerpts RealSR/DRealSR at ×4; fixed-scale methods and frozen ASSR plus a proxy follow different training protocols, so the ranking must not be read as a strictly compute-matched comparison.

Method	RealSR PSNR / SSIM / LPIPS	DRealSR PSNR / SSIM / LPIPS
SynDiff	25.2461 / 0.7588 / 0.2371	28.0125 / 0.8136 / 0.1739
RealDGen	26.1615 / 0.7940 / 0.2226	28.6629 / 0.8360 / 0.1497
LIIF + CDR	26.8051 / 0.7822 / 0.1943	28.7324 / 0.7743 / 0.2367
CiaoSR + CDR	26.9161 / 0.7842 / 0.1910	28.8057 / 0.7728 / 0.2339
HIIF + CDR	26.7693 / 0.7822 / 0.1964	28.6953 / 0.7704 / 0.2382

On RealSR ×4, CiaoSR + CDR's PSNR is 0.7546 dB higher than RealDGen's with lower LPIPS, but lower SSIM. On DRealSR the PSNR is slightly higher while SSIM and LPIPS both trail RealDGen. Relative to bare CiaoSR, DRealSR gains 0.7349 dB PSNR and 0.0936 LPIPS, yet SSIM drops by 0.0313. There is therefore no evidence of across-the-board improvement on all three metrics across both datasets.

The paper's largest PSNR gain of 4.6858 dB comes from CiaoSR on RealSR ×2, where the result is 34.9864 dB — not from the ×4 setting in the table above. Across COZ's eight scales, the PSNR gains of the three backbones over their respective bare models range from 3.4634 to 3.9622 dB; gains persist at ×5 and ×6 beyond the training range, but they only speak for the measured interval.

Ablation Study

The following excerpts representative scales from the paper's Table 3, all COZ PSNR. The difference between fixed-factor-trained variants and the fully continuous conditioning system is a set of training designs; the table alone cannot prove that any latent variable is strictly disentangled.

Setting	×2	×3	×4	×6
Bare LIIF	27.8485	26.0925	24.7187	22.7951
Fixed ×2 training	28.2575	26.0842	24.8292	22.8876
Fixed ×3 training	28.2483	26.0892	24.8364	22.8996
Fixed ×4 training	28.2358	26.3711	24.8576	22.9156
Full continuous-scale model	31.7725	29.9801	28.6130	26.4225

Fixed ×2 training beats bare LIIF by 0.4090 dB at its own factor; fixed ×3 training is 0.0033 dB lower at its own factor; fixed ×4 training is 0.1389 dB higher. Fixed-factor training therefore does not guarantee a positive gain even at its own factor. Relative to the best of the three fixed variants, the full model is higher by 3.5150, 3.6090, 3.7554, and 3.5069 dB at ×2, ×3, ×4, and ×6 respectively.

The paper's Table 4 compares fusion location and granularity; K_d is generated from degradation features alone, while K_dc uses both degradation and scale features. The rows shown here are the ones that directly test the final design:

Fusion strategy	×2 PSNR	×3 PSNR	×4 PSNR
Feature concatenation	27.7121	26.8999	26.1973
Global modulation K_dc	28.3928	26.9515	26.4270
Channel-wise modulation K_d	28.2760	27.0157	26.5597
Channel-wise modulation K_dc (full)	31.7725	29.9801	28.6130

Key Findings

• Continuous conditioning beats these fixed variants. The advantage at representative scales is about 3.5–3.8 dB, which supports the practical value of cross-scale modeling but does not prove that scale entanglement is the sole bottleneck of real-world degradation.
• Kernel generation plus channel-wise modulation matters. The full configuration is 3.3797 dB above global K_dc at ×2; this is a configuration-level ablation — the numbers alone cannot assert that an accurate optical frequency band was learned.
• Metric trade-offs persist. On COZ all three metrics improve over the bare backbones, but SSIM drops on DRealSR; the largest gains from different tables must not be stitched together as if they came from one setting.

Highlights & Insights

• Reuses existing backbones. Adapting the input distribution instead of retraining every ASSR model gives deployed models another path into real-world imagery, with the cost shifted to proxy inference.
• Positives and negatives explicitly control what the model attends to. Different images at the same scale are positives and the same image at different scales are negatives, directly encoding the variation to keep and the variation to ignore — more concrete than a generic demand for "disentanglement".
• The scale participates in generating the conditional kernel. Rather than concatenating a bare zoom value, the method changes the filter applied to the content, adding structural constraints to the diffusion condition.

Limitations & Future Work

• Inference cost is not adequately quantified. Multiple encoders plus a DDPM are more complex than a direct ASSR forward pass; without latency, parameter counts, and FLOPs, the deployment benefit is hard to judge.
• Reference-based transfer rests on empirical decoupling. Whether content-unrelated references remain stable, and whether they mismatch under sensor-specific noise, lacks a systematic sensitivity analysis.
• Conclusions are confined to the tested range. Working at the extrapolated ×5 and ×6 does not make every factor reliable, and DRealSR's structural and perceptual metrics are not comprehensively ahead.
• Mechanism claims outrun proof. The paper's "strictly invariant", "orthogonal", and "complete manifold" wording should be read as design goals; the corrupted formulas in the cache also limit exact reproduction.

Related Work & Insights

• vs RealDGen: This paper compares against its fixed-scale generate-pairs-then-retrain scheme and replaces it with inference-time input rectification; RealDGen still wins LPIPS on DRealSR, so no blanket claim of full replacement is warranted.
• vs real continuous-scale datasets: COZ provides continuous-factor evaluation; this paper avoids relying on real paired training, which does not mean real data is no longer needed for other tasks.
• vs Real-ESRGAN: High-order synthetic degradations remain the source of this paper's training data — CDR learns a conversion on top of them rather than fully escaping handcrafted degradation assumptions.
• vs ZSSR: ZSSR trains self-supervised on the test image itself; this paper uses a pre-trained proxy with all models frozen at test time, so the meaning of "zero-shot" differs.

Rating

• Novelty: 4/5. Input-side rectification combined with continuous-scale conditioning is clearly framed, though the underlying components are not newly proposed.
• Experimental Thoroughness: 3/5. Multiple backbones, multiple benchmarks, and two kinds of ablations are valuable; latency and compute-matched adaptation comparisons are missing.
• Writing Quality: 3/5. The pipeline is clearly laid out, but the strict-decoupling wording and metric summaries need calibration.
• Value: 4/5. Offers a path to reuse synthetically pre-trained ASSR models; inference cost and robustness to real degradations still need verification.