ECCV2026 Image Restoration Multimodal VLM Image Super-Resolution Reward Model Preference Alignment GRPO Multimodal Large Language Model

RefReward-SR: LR-Conditioned Reward Modeling for Preference-Aligned Super-Resolution¶

Conference: ECCV2026
Paper: ECCV page
Area: Image Restoration / Multimodal VLM
Keywords: Image Super-Resolution, Reward Model, Preference Alignment, GRPO, Multimodal Large Language Model

TL;DR¶

Treating the low-resolution image as a semantic anchor, the paper fine-tunes Qwen3-VL 8B with GRPO into an LR-conditioned, interpretable reward model (RefReward-SR) backed by the first large-scale LR-conditioned preference dataset (RefSR-18K), and then uses that reward as the core signal in a GRPO fine-tuning of the diffusion SR model C-FLUX — reaching 85.0% in-domain agreement with human annotators, on par with the 84.7% agreement among the annotators themselves.

Background & Motivation¶

Real-world super-resolution (Real-ISR) has moved from bicubic degradation to complex, unknown degradations: BSRGAN and Real-ESRGAN framed the task as distribution matching with adversarial losses, and diffusion-based methods such as StableSR, DiffBIR, SeeSR and OSEDiff leverage generative priors to synthesize fine details even under severe degradation. The stronger the generative models become, however, the more the evaluation and optimization frameworks lag behind. Full-reference (FR) metrics — PSNR, SSIM, LPIPS — rely on strict spatial correspondence, which is overly conservative for generative SR: they penalize "reasonable hallucinations", i.e. details that are semantically sound and visually enriching. No-reference (NR) metrics such as MUSIQ, CLIPIQA and TOPIQ-IAA need no reference and excel at low-level sharpness and textural naturalness, but precisely because they lack a reference they cannot constrain content fidelity, so a model can inflate its score through over-sharpening, unfaithful artifacts, or even by altering the subject identity. More fundamentally, both families reflect high-level semantic plausibility only passively through low-level cues, ignoring the top-down nature of human perception, which first confirms semantic consistency and only then scrutinizes low-level detail. As a result, they cannot explicitly penalize semantic violations and structural distortions that contradict world knowledge.

The second problem lies on the optimization side. Most SR methods maintain structure with supervised losses, and some add adversarial losses, but strictly fitting the empirical GT distribution is not the same as capturing higher-level human semantic preferences; moreover, real-world GT itself often carries degradations, so an overly GT-dependent model reproduces those imperfections instead of improving quality. Text-to-image has long used RLHF for systematic preference alignment, yet that route remains limited in SR: prior attempts either reuse existing metrics as rewards — metrics that are themselves flawed — or call general-purpose MLLMs that are not well suited to low-level vision, so neither fully unlocks the potential of reinforcement learning for SR.

The angle of this paper is: since SR is an ill-posed inverse problem where one LR image legitimately admits several plausible HR reconstructions, let the LR image be the single semantic anchor and recast SR evaluation as an LR-conditioned preference modeling problem — not "does the reconstruction look like the GT", but "is it faithful to the structures and semantic cues of the LR, and is the reconstructed content natural and free of distortion-induced anomalies". Core idea: first fine-tune an MLLM with GRPO on human pairwise preferences (RefSR-18K) into an LR-conditioned, interpretable ranking reward model, then feed that reward as the core of a composite reward into GRPO to directly optimize the generation distribution of a diffusion SR model, aligning with human semantic preferences while keeping structural fidelity.

Method¶

Overall Architecture¶

The method has two stages joined by a single reward model. Stage one builds the evaluator: images are sampled from LSDIR, LR inputs are synthesized with the SeeSR degradation pipeline, eight state-of-the-art SR models generate candidate HR images alongside the GT, and several annotators rank the candidates within each group by two criteria — LR–HR semantic consistency and semantic plausibility — producing RefSR-18K. A format reward plus an LR-conditioned rank reward then fine-tune Qwen3-VL 8B with GRPO into RefReward-SR, a scorer that emits both a <thinking> reasoning chain and a scalar score and whose judgments align with human preferences. At evaluation time, to keep local differences from being masked by similar overall structures, a global–local crop scoring layer is stacked on top: RAM and Grounding DINO select representative regions, and global and crop scores are fused by area-weighted averaging. Stage two puts the reward to work: fine-tuning C-FLUX (from DP2O-SR) is cast as an RL problem, a group of reconstructions is sampled per LR input, and a composite reward of RefReward-SR semantic reward + LPIPS perceptual penalty + DEQA quality reward drives within-group relative optimization, yielding the preference-aligned HR image.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["LR input + HR candidates"] --> B["LR-conditioned preference data RefSR-18K<br/>consistency + plausibility ranking"]
    B --> C["LR-conditioned rank reward<br/>Thurstone comparison → continuous fidelity"]
    C --> D["Global–local crop scoring<br/>RAM + Grounding DINO → area weighting"]
    D --> E["Composite-reward SR alignment<br/>Ref + LPIPS + DEQA → GRPO"]
    E --> F["Preference-aligned HR reconstruction"]
    C -.->|global score only during alignment| E

Key Designs¶

1. LR-conditioned preference data RefSR-18K: giving "good" a semantic anchor to stand on

Existing IQA and aesthetic datasets (KADID, SPAQ, Pick-a-Pic, HPS and the like) are either single-image scoring that only captures low-level distortions such as noise, or global artistic appeal; they carry no LR reference with which to judge whether "the generated details are faithful to the semantics of the input", and they provide no explicit ranking of the generative anomalies specific to SR — structural distortion, unreasonable textures, localized content hallucination. RefSR-18K fills this gap. 5,130 high-quality images covering a wide range of categories are curated from LSDIR, and LR inputs are synthesized by random cropping plus the complex degradation pipeline of SeeSR. The HR candidates come from eight state-of-the-art SR models (DiffBIR, SeeSR, OSEDiff, S3Diff, LucidFlux, DiT4SR, OMGSR-S, HYPIR) — a pool deliberately spanning both single-step and multi-step generation paradigms and the two distinct generative priors of Stable Diffusion and FLUX — plus the GT. Each annotation group consists of one LR reference and four HR candidates randomly drawn from the nine sources, and annotators rank the four jointly by two criteria: (1) LR–HR semantic consistency, i.e. whether the generated details faithfully follow the structures and semantic cues present in the LR; and (2) semantic plausibility, i.e. whether the reconstructed objects are semantically natural and free of distortion-induced anomalies. Ties are allowed when quality differences are indistinguishable. Group rankings are then converted into pairwise preferences, used both to train RefReward-SR and to measure preference alignment. Because annotation in image restoration is subjective and fine-grained, quality control is strict: annotators must pass training and qualification tests, and only those highly consistent with expert judgments are retained (roughly 50% are filtered out); each group is independently annotated by at least three qualified annotators and aggregated by average rank; finally an expert review and post-hoc filtering removes groups whose candidates are nearly indistinguishable, whose images lack enough semantic content, or where annotators disagree substantially. This process retains 4,699 high-quality annotation groups, i.e. 18,796 annotated HR reconstructions paired with LR references. This dataset is the foundation of the whole pipeline — without it, neither the reward model nor the alignment optimization would be possible.

2. Thurstone-based LR-conditioned rank reward: turning a ranking judgment into a continuous reward

A reward model easily degenerates into a black-box scorer: if it only emits an absolute score, the score is neither interpretable nor stable across rollouts, which makes ranking unreliable. RefReward-SR instead lets the policy sample K scores for the same candidate under the same (LR, prompt) conditioning, and uses the sample mean and variance of these K predictions to build a Thurstone comparative probability — the probability that the model considers candidate \(x_i\) better than another candidate \(x_j\) in the same group. The variance term is explicit, so the model's own uncertainty about an image is encoded into the comparison:

\[p_k(x_i,x_j)=\Phi\!\left(\frac{q_k(x_i)-\mu(q(x_j))}{\sqrt{\sigma^2(q(x_i))\,\sigma^2(q(x_j))}+\gamma}\right)\]

where \(\Phi\) is the standard normal cumulative distribution, \(\mu(\cdot)\) and \(\sigma^2(\cdot)\) are the sample mean and variance of the K predictions, and \(\gamma\) is a small stabilizing constant. The reward itself is not a 0/1 verdict on whether the ordering is right, but a continuous fidelity measure between the predicted probability and the true human preference \(p(x_i,x_j)\in\{1,0.5,0\}\) (win/tie/loss), averaged over the remaining \(G-1\) candidates in the group:

\[R_k(x_i)=\frac{1}{G-1}\sum_{j\neq i} d\big(p(x_i,x_j),\,p_k(x_i,x_j)\big)\]

According to the paper, \(d\) is formed by the two radicals \(\sqrt{p\,p_k}\) and \(\sqrt{(1-p)(1-p_k)}\), i.e. a Bhattacharyya/Hellinger-style fidelity; ⚠️ the cached text has this equation broken up by OCR with the connecting symbol between the two terms lost, so the exact form is not reconstructed here — refer to the original paper. The advantage of a continuous reward is that it captures subtle variations in the quality ordering and penalizes outlier predictions most heavily: putting the human-ranked worst image first drops the reward sharply, whereas a discrete ranking reward provides no such gradient. A format reward accompanies it: the output must first contain a reasoning chain inside <thinking>…</thinking> that states which problems the HR image has and where, conditioned on the LR reference, and must then place the numerical rating inside <answer>…</answer>; if any of these formats is violated the format score is 0. This hard constraint is precisely what makes RefReward-SR an interpretable "reason first, then score" evaluator rather than a black box.

3. Global–local crop scoring: closing the blind spot when differences live in local details

In practice different SR results share very similar overall structures and differ mainly in local details, so global scoring alone easily misses the decisive nuances; human annotators, by contrast, first look at the whole image and then compare the primary objects one by one. The proposed strategy mimics this in two stages. Region proposals come first: RAM extracts semantic tags, the tags guide Grounding DINO to produce candidate bounding boxes, and a bounding-box filtering pipeline selects representative regions that are low-overlap, texture-rich and salient (the detailed rules are in the supplementary material). Joint evaluation comes second: the MLLM scores the global image and each selected crop separately, and the multi-scale assessments are fused into a final score by area-weighted averaging:

\[S_{\text{final}}=\frac{A_g S_g+\sum_{i=1}^{N}A_i S_i}{A_g+\sum_{i=1}^{N}A_i}\]

where \(S_g\), \(A_g\) are the score and area of the global image and \(S_i\), \(A_i\) those of the \(i\)-th local crop. The area weights let larger regions influence the final score more, covering both global information and local detail. One practical trade-off is worth noting: the strategy is used only at evaluation time. During GRPO fine-tuning of the SR model only the global score is used, because the global score already provides strong enough semantic guidance while scoring many crops would multiply the reward cost.

4. Composite-reward SR alignment: making the reward actually enter the generation process

With a reliable evaluator in hand, fine-tuning the C-FLUX diffusion SR model is formulated as a reinforcement learning problem: given an LR input, the conditional policy samples a group of G reconstructions at once and optimization relies only on relative quality comparisons within the group — exactly the setting GRPO addresses, with no critic and with group-normalized relative advantages. Reward design is the decisive factor here, and a single reward is clearly insufficient. What goes wrong with RefReward-SR alone? It governs high-level semantics only: it neither constrains pixel-level fidelity nor specifically suppresses blur and noise, so the model could plausibly drift structurally in exchange for semantic plausibility. The reward therefore has three complementary components:

\[R_{\text{total}}(y_i)=\lambda_1 R_{\text{Ref}}(y_i,x_{LR})-\lambda_2 R_{\text{LPIPS}}(y_i,y_{GT})+\lambda_3 R_{\text{DEQA}}(y_i)\]

The RefReward-SR reward is the core semantic signal, judging both the consistency between the generated image and the LR and the plausibility of the content itself. LPIPS decreases with higher perceptual similarity to the GT, so it enters the total reward as a negative penalty term that keeps the structure from deviating too far and preserves perceptual fidelity. DEQA serves as an image quality assessment reward that raises overall sharpness and strengthens the penalization of low-level degradations such as blur and noise. The weights \(\lambda_1,\lambda_2,\lambda_3\) balance semantic human preference alignment, perceptual fidelity and artifact suppression respectively (⚠️ the three terms of Eq. 5 are run together by OCR in the cached text; the signs here follow the prose description, where LPIPS is explicitly a negative penalty term, and the values of the λ's are not given in the paper).

Loss & Training¶

Both stages use GRPO. In the first stage the policy is Qwen3-VL 8B and the reward is the format reward plus the LR-conditioned rank reward over RefSR-18K group rankings; the goal is to align the model's judgments with human preferences while letting the reasoning chain emerge from RL rather than being imposed by supervised fine-tuning. In the second stage the policy is the C-FLUX diffusion model: a group of reconstructions is sampled per LR input and optimized relatively within the group under the composite Ref + LPIPS + DEQA reward. The SR training data is synthesized from LSDIR with Real-ESRGAN degradation at 512×512 to match the base model C-FLUX. Evaluation follows the StableSR protocol: the synthetic set is DIV2K-Val, where 3,000 images are randomly cropped to 512×512 and LR inputs are produced by the same Real-ESRGAN degradation, while the real-world set is RealSR with 100 paired LR–HR images at 128×128 and 512×512. ⚠️ The paper does not report the group size G, the number of rollouts K, or the concrete values of \(\lambda_1/\lambda_2/\lambda_3\); these live in the supplementary material or are omitted — refer to the original paper.

Key Experimental Results¶

Main Results¶

The experiments have two parts: first, how well the reward model itself agrees with human preferences; second, whether the SR model aligned with it is genuinely better. The evaluation protocol follows ImageReward, using agreement (how often the model's preference matches the human annotation), Recall@1 (how often the human-best image is ranked first by the model) and Filter@1 (how often the human-worst image is ranked last). Two 200-group test sets are built: in-domain (candidates from the eight models seen in training plus the GT) and out-of-domain (two candidates generated by the entirely unseen TSD-SR and PiSA-SR).

Method	In-domain Agreement (%)	In-domain Recall@1	Out-of-domain Agreement (%)	Out-of-domain Recall@1
Annotators	84.7 ± 1.4	-	81.8 ± 1.3	-
PSNR	41.8 ± 1.6	45.0	35.5 ± 2.3	25.5
SSIM	51.8 ± 2.0	51.0	43.3 ± 1.8	26.0
LPIPS	62.5 ± 0.9	49.5	58.8 ± 2.0	44.5
MUSIQ	60.9 ± 1.4	40.5	57.6 ± 2.7	38.0
CLIPIQA	54.0 ± 1.7	31.0	56.9 ± 2.5	32.0
Q-Align	54.9 ± 1.7	35.0	54.3 ± 1.0	40.5
VQ-R1	47.8 ± 1.3	49.0	44.1 ± 2.8	50.5
GPT-5.2	52.5 ± 1.0	30.7	49.7 ± 2.2	38.1
Gemini 3 Pro	58.8 ± 1.5	54.0	47.9 ± 2.1	42.6
Qwen3-VL 8B (zero-shot)	31.8 ± 0.4	67.5	26.1 ± 0.9	59.0
RefReward-SR (ours)	85.0 ± 2.4	84.5	80.2 ± 2.2	77.0

(⚠️ The GPT-5.2 and Gemini 3 Pro entries in the paper cite the GPT-4 and Gemini technical reports respectively; the model version labels are as given in the original.) The method also leads on Filter@1: 78.0 in-domain and 73.0 out-of-domain, the highest among all methods.

The aligned SR model is then compared with a suite of state-of-the-art Real-ISR methods on RealSR and DIV2K-Val, with the base model C-FLUX and the DPO-fine-tuned DP2O-FLUX listed separately as controls for how much the alignment strategy itself contributes:

Dataset	Method	PSNR↑	SSIM↑	LPIPS↓	MUSIQ↑	CLIPIQA↑	Q-Align↑	VQ-R1↑
RealSR	SeeSR	25.14	0.7211	0.3007	69.82	0.6705	3.7190	3.9830
RealSR	PiSA-SR	25.50	0.7418	0.2672	70.15	0.6698	3.6354	4.0180
RealSR	TSD-SR	23.40	0.6938	0.2823	71.26	0.7416	3.8444	4.0800
RealSR	C-FLUX (base)	24.42	0.6742	0.3376	68.70	0.6221	3.4943	3.9650
RealSR	DP2O-FLUX (DPO baseline)	24.51	0.6774	0.3259	72.42	0.7156	3.9757	4.2170
RealSR	Ours	24.68	0.7101	0.3202	71.70	0.7007	4.1596	4.2930
DIV2K-Val	SeeSR	23.68	0.6043	0.3194	68.68	0.6936	3.9770	4.2013
DIV2K-Val	PiSA-SR	23.87	0.6058	0.2823	69.68	0.6929	3.8804	4.2054
DIV2K-Val	TSD-SR	22.41	0.5644	0.2710	71.64	0.7559	4.0147	4.1529
DIV2K-Val	C-FLUX (base)	22.71	0.5520	0.3349	69.70	0.6781	4.0573	4.2558
DIV2K-Val	DP2O-FLUX (DPO baseline)	22.72	0.5536	0.3216	72.98	0.7536	4.3983	4.4343
DIV2K-Val	Ours	22.88	0.5786	0.3260	71.95	0.7302	4.4605	4.4396

A user study guards against reward hacking from evaluating one's own model with one's own reward: 10 images are randomly drawn from DIV2K-Val and 10 from RealSR, and 15 volunteers with image-processing experience pick the better reconstruction. The win rate of the proposed method is 72.7%, above DP2O-FLUX (67.1%) and C-FLUX (60.2%), and the average RefReward-SR score correlates strongly with the win rate (2.09 / 1.80 / 1.77 for the three). The PiSA-SR and TSD-SR entries included for reference score and win lower (about 1.63 / 1.40 and roughly 31.9% / 18.1%; ⚠️ the mapping between numbers and methods in Fig. 3 is as given in the original).

Ablation Study¶

On the reward-model side (agreement in %), scaling the training data from 1K to 2K groups and then to the full 4,699 raises in-domain agreement monotonically, and removing global–local crop scoring hurts the out-of-domain set more than the in-domain one.

Config	In-domain Agreement (%)	Out-of-domain Agreement (%)	Note
1K data	81.2 ± 1.9	76.6 ± 3.2	trained on 1K groups, weakest generalization
2K data	82.7 ± 1.9	77.4 ± 1.8	doubling the data gives a consistent gain
w/o global–local crops	83.6 ± 2.6	78.1 ± 2.3	crop scoring removed, −2.1 out-of-domain
Full model	85.0 ± 2.4	80.2 ± 2.2	data scale + crop scoring together

On the SR side (RealSR), each reward term of Eq. 5 is removed in turn:

Config	PSNR↑	SSIM↑	LPIPS↓	MUSIQ↑	CLIPIQA↑	Q-Align↑	Note
C-FLUX (base)	24.42	0.6742	0.3376	68.70	0.6221	3.4943	unaligned base model
w/o RefReward-SR	23.89	0.6920	0.3211	73.48	0.7239	4.1656	FR barely drops, some NR even rises, but semantics distort and structures twist
w/o LPIPS	21.98	0.5798	0.4026	74.55	0.7434	4.3521	low-level fidelity collapses while NR peaks — textbook metric gaming
w/o DEQA	23.87	0.7099	0.2967	69.02	0.6153	3.5313	clarity drops, details blur
Full model	24.68	0.7101	0.3202	71.70	0.7007	4.1596	best balance across metrics

Key Findings¶

Fine-tuning is the precondition for the reward model to work. Zero-shot Qwen3-VL 8B agrees with humans only 31.8% of the time, while the fine-tuned RefReward-SR reaches 85.0%, a 53.2-point gain that matches the annotators' own 84.7%. LR-conditioned pairwise preferences plus GRPO genuinely close the fine-grained SR evaluation gap of general-purpose MLLMs.
Zero-shot MLLMs fail in a peculiar way: when candidates have similar structures they are insensitive to subtle differences and tend to give identical scores, which under this paper's computation protocol pushes agreement below 50% while paradoxically inflating Recall@1 and Filter@1 (Qwen3-VL 8B has 31.8% agreement but 67.5% Recall@1). Recall@1 alone can therefore be fooled by a "always tie" strategy; agreement and Filter@1 must be read together.
Low-level metrics are not useless: apart from PSNR, every method with agreement below 50% is an MLLM, and FR metrics such as LPIPS do indirectly reflect semantic preference to a considerable degree (62.5% in-domain) — though far below the 85.0% reported here.
The three rewards play distinct, non-substitutable roles. Removing LPIPS is by far the most damaging: PSNR falls from 24.68 to 21.98, SSIM from 0.7101 to 0.5798 and LPIPS degrades from 0.3202 to 0.4026, yet NR metrics such as MUSIQ/CLIPIQA/NIMA climb to their highest values — a vivid demonstration of NR metric gaming. Without RefReward-SR the FR metrics decline only slightly, since fidelity is mainly held by the LPIPS reward, and some NR metrics even improve, but the outputs show semantic distortion and structural twisting. Without DEQA the affected NR metrics drop clearly (MUSIQ 71.70→69.02, Q-Align 4.1596→3.5313) and details blur. Only the full configuration is balanced across all metrics.
Global–local crop scoring mainly buys generalization: removing it costs 1.4 in-domain and 2.1 out-of-domain, showing that it matters especially for capturing fine-grained semantic inconsistencies, in particular for unseen SR models.
One striking contradiction: C-FLUX performs worse than TSD-SR and PiSA-SR on conventional NR/FR metrics yet earns a higher human win rate. This corroborates the reward-model evaluation — existing metrics are indeed insufficient for measuring human semantic preference.
Against DP2O-FLUX, which is also RL-fine-tuned (with DPO), the method leads on most metrics (PSNR 24.51→24.68, SSIM 0.6774→0.7101, LPIPS 0.3259→0.3202, Q-Align 3.9757→4.1596, VQ-R1 4.2170→4.2930) but is slightly lower on MUSIQ (72.42 vs 71.70) and CLIPIQA (0.7156 vs 0.7007), so it is not a clean sweep.

Highlights & Insights¶

Using the LR image rather than the GT as the conditioning anchor is the most elegant move in the paper. SR is ill-posed, so one LR image legitimately corresponds to several plausible HR reconstructions; conditioning on the LR sidesteps both the strict spatial-correspondence demands of FR metrics and the reference-free drift of NR metrics. The idea transfers directly to deblurring, deraining and inpainting — any restoration task where a degraded input carries the semantics can use that input as the anchor.
Writing the reward as a continuous fidelity through a Thurstone model with multi-rollout variance: instead of a 0/1 verdict on whether the ordering is correct, the reward measures the fidelity between the predicted distribution and the human preference distribution, with the variance term explicitly modeling uncertainty and the heaviest penalty reserved for outlier predictions. Compared with VQ-R1-style "learn to rank via RL", the extra ingredient is that the reward itself is continuous, which yields a finer-grained gradient.
A format reward that turns the reasoning chain into a hard constraint: inducing reasoning through RL rather than supervision makes "locate the problem, then score" the model's default behavior. The qualitative examples in the paper (1.00 for a structurally distorted gear, 3.80 for a successful reconstruction) show the reasoning actually localizes to specific regions — rare interpretability for low-level vision evaluation.
Global–local crops with area weighting is a nearly free, general trick that any MLLM scorer could adopt; even more instructive is the deliberate decision to drop it during RL fine-tuning in exchange for efficiency — evaluation may be slow, training may not.
The strong correlation between the user study and the average reward scores (72.7%/67.1%/60.2% against 2.09/1.80/1.77) is a neat act of self-verification: the reward model has not become a gameable metric, and the reward really does steer generation toward what humans prefer.

Limitations & Future Work¶

Missing key hyper-parameters hamper reproduction: \(\lambda_1,\lambda_2,\lambda_3\), the sampling group size G, the rollout count K, and the crop count and filtering rules are all absent from the main text and deferred to the supplementary material.
Out-of-domain generalization adds only two unseen models (TSD-SR and PiSA-SR), and out-of-domain agreement drops overall (even for human annotators). The authors attribute this to the unseen models producing outputs of quality very similar to the other candidates, making discrimination intrinsically harder, but whether the reward model stays reliable for entirely new generative paradigms (stronger diffusion transformers, autoregressive SR) remains open.
On reward hacking: the human user study circumvents the self-evaluation problem of scoring one's own model with one's own reward, but the paper does not report whether the training reward and the human win rate rise together, nor whether reward saturation or KL drift occurs — a notable gap for a method that aligns via RL.
Only one diffusion backbone (C-FLUX) and one reward backbone (Qwen3-VL 8B) are validated; whether the recipe transfers to single-step models such as OSEDiff or to other MLLMs is unknown.
"Semantic plausibility" depends on the annotators' subjective world knowledge; its stability across cultures and domains (medical imaging, remote sensing) — which are important SR application areas — is not analyzed.
A natural extension: the <thinking> chain of RefReward-SR is itself a localized failure diagnosis, usable to automatically taxonomize the failure modes of SR models or, conversely, as a quality filter for training data — considerably more valuable than a scalar score alone.

vs traditional FR/NR metrics (PSNR/SSIM/LPIPS and MUSIQ/CLIPIQA/Q-Align/TOPIQ-IAA): the former demand strict spatial correspondence and penalize plausible hallucinations, the latter have no reference and cannot anchor semantics; both reflect high-level semantics only passively through low-level cues. This paper recasts evaluation as an LR-conditioned preference judgment, moving from distortion measurement to preference judgment and lifting in-domain agreement from 62.5% (the strongest baseline, LPIPS) to 85.0%.
vs general-purpose MLLM evaluators (GPT-5.2 / Gemini 3 Pro / zero-shot Qwen3-VL): zero-shot MLLMs have too coarse a score resolution and assign identical scores to candidates with similar structures, so fine-grained ranking fails and agreement falls below 50%. Pairwise preferences from RefSR-18K plus GRPO lift an 8B model to human-level granularity, at a far smaller parameter budget than the closed-source baselines.
vs generic IQA / preference reward models (ImageReward, DEQA, VQ-R1): ImageReward-style rewards are single-image, reference-free aesthetic predictors and inherently cannot answer whether the generated details are faithful to the input; VQ-R1 applies RL ranking to IQA but still in a no-reference setting. This paper's reward is explicitly conditioned on the LR and judges semantic consistency plus plausibility, while DEQA is demoted to a supplementary quality term that suppresses blur and noise.
vs RL attempts on SR (DSPO, RealSR-R1, IRPO, DP2O-SR): DSPO-style methods reuse existing metrics as rewards, while RealSR-R1 and IRPO score directly with general-purpose MLLMs; neither solves the mismatch between the reward and low-level vision. DP2O-SR uses DPO-style direct preference optimization. Against its closest relative DP2O-SR, this paper differs by training a dedicated LR-conditioned reward model and sampling online under GRPO rather than relying on fixed preference pairs, leading the DP2O-FLUX variant on most metrics while additionally providing an interpretable evaluator.

Rating¶

Novelty: ⭐⭐⭐⭐ First LR-conditioned preference dataset and reward model for SR; the "LR as semantic anchor" reframing is solid, though GRPO, the Thurstone model and RL fine-tuning of diffusion models are all existing building blocks — a combinatorial contribution.
Experimental Thoroughness: ⭐⭐⭐⭐ Evaluator alignment, in-domain/out-of-domain generalization, main SR results, a user study and two ablations are all covered, and the dual test-set design is particularly rigorous; points off for missing key hyper-parameters and crop details, and for validating only one SR backbone.
Writing Quality: ⭐⭐⭐⭐ The motivation is concrete and the two failure modes are clearly dissected; however the equations are badly damaged by OCR in the cached text and some symbols (\(\gamma\), the connector in the reward equation) are under-explained, making exact reproduction hard.
Value: ⭐⭐⭐⭐ Provides a reusable and transferable "LR-conditioned reward + GRPO alignment" recipe, and the reward model doubles as a practical SR evaluation tool — a meaningful push for preference alignment in low-level vision.