EditHF-1M: A Million-Scale Rich Human Preference Feedback for Image Editing¶
Conference: ECCV2026
Paper: ECCV Paper
Code: https://github.com/IntMeGroup/EditHF
Area: Image Generation
Keywords: image editing, human preferences, multidimensional quality assessment, reward model, direct preference optimization
TL;DR¶
EditHF-1M trains the unified evaluator EditHF using three-dimensional human rankings and scores over 1.01M edited images, then uses its selected preference pairs to optimize Qwen-Image-Edit, raising the editor's human overall score on this dataset from 75.39 to 84.26.
Background & Motivation¶
Successful image editing requires more than an attractive output: evaluation must identify the requested change, the change that occurred, and the content that should have remained untouched. An output can look polished while ignoring the instruction, or replace the requested object while damaging identity, structure, and background. Traditional image quality metrics mainly recognize distortions, while text-to-image metrics mainly measure correspondence between text and the output; neither alone adequately captures this conditional change. General multimodal models can understand images and instructions, but a zero-shot response is not necessarily a stable estimate of human preference.
Existing editing benchmarks increasingly include human opinion scores or pairwise preferences, but often cover limited source images, editing tasks, or generators. An evaluator exposed to errors from only a few generators may learn their styles rather than transferable editing quality criteria. Supervision formats also differ: comparing two outputs under the same input is often more reliable than assigning absolute scores, but relative order does not inherently establish a common scale across inputs. Absolute ratings enable broader comparisons, yet can be less sensitive to subtle differences between similar candidates in one group. The paper therefore expands both data coverage and feedback types, rather than simply adding another small leaderboard.
The authors connect a dataset, an evaluator, and editor optimization: collect human feedback on outputs from multiple models, learn to score and rank them, and test whether these judgments improve editing. The dataset spans 43 tasks and 23 editing models, with separate annotations for visual quality, editing alignment, and attribute preservation. Evaluation and generation guidance can consequently share supervision, but still require separate validation: high evaluation correlation does not automatically establish effective post-training. Core Idea: use large-scale within-group preferences to establish relative quality, a smaller set of absolute ratings to calibrate scores, and the resulting three-dimensional evaluator to select preference data for an editor.
Method¶
Overall Architecture¶
The paper introduces three distinct objects: EditHF-1M is a human-feedback dataset, EditHF is the learned evaluator, and EditHF-Reward uses that evaluator to construct preference data for optimization. The pipeline builds supervision through “Multi-source dual feedback,” creates a “Three-input scorer,” learns three-dimensional scoring through “Staged joint training,” and proceeds to “Reward-guided editing.” The evaluator reads a source image, an edited image, and an editing prompt to produce three scores; it does not generate edited images itself. For editor optimization, it first ranks multiple candidates and forms chosen/rejected pairs, which train a separate image editing model.
%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
Inputs["Source images, prompts<br/>multi-model edits"] --> Data["Multi-source dual feedback"]
Data -->|Human score and preference supervision| Scorer["Three-input scorer"]
Scorer --> Training["Staged joint training"]
Training -->|Trained EditHF| Reward["Reward-guided editing"]
Candidates["New candidate group<br/>source image and prompt"] -->|Inference-time scoring| Reward
Reward -->|Preference pairs and DPO| Output["Refined editing model"]
Key Designs¶
1. Multi-source dual feedback: assign complementary roles to relative preferences and absolute scores
The data cover global-level, object-level, human-centric, and low-level editing, rather than only common addition, removal, or style-transfer operations. For photography website images, InternVL3.5 generates prompts from the content and task, followed by human inspection and revision. Instruction-based prompts from Pico-Banana-400K receive complementary description-based versions so that editors with different interfaces can be compared. Source images and prompts for low-level tasks come from EBench-18K; the main text reports final totals of 44.3K source images and 1.01M edited images. The authors filter abnormal outputs such as black or noisy images and state that the test set maintains comparable image sets across tasks and models. The reported scale therefore describes constructed and filtered data, not a raw invocation log retaining every model failure. The initial source counts in the text cannot simply be added to reconstruct the final total; this note retains the final statistics without guessing intermediate deduplication steps.
Ranking groups contain outputs sharing the same source image and prompt, with independent rankings for each dimension instead of one ordering that conceals different errors. Visual quality covers realism, artifacts, structure, and detail; editing alignment checks whether requested changes were completed correctly; attribute preservation checks identity, key characteristics, and context outside intended edits. Each group is judged by 3 professional annotators; low-agreement groups are reassessed by another 3 annotators, and average ranks produce within-group preference pairs. The 29.1M preferences are therefore the three-dimensional total expanded from group rankings, not the same number of independently performed human pairwise choices. Scoring covers 49.4K edited images, with 5 raters per training score and 15 per test score, yielding approximately 148K aggregated scores across the three dimensions. Continuous ratings undergo outlier removal beyond 2 standard deviations, removal of participants with more than 5% outlier ratings, and then per-rater standardization and range scaling. The authors also reject annotations whose scores and preferences disagree, encouraging compatible supervision but potentially suppressing genuine disagreement. Equation (1) is corrupted in the cached extraction, preventing reliable recovery of the full score aggregation formula; only processing steps confirmed by the prose are retained here.
2. Three-input scorer: jointly assess image content, editing changes, and instruction constraints
An edited image alone cannot reveal which changes were allowed; even an image plus a target description can omit source attributes that should have been preserved. EditHF therefore encodes both source and edited images with an InternViT-based vision encoder and maps the features into language-space visual tokens through a trainable projector. The editing prompt becomes text tokens, which enter InternVL3.5 together with the two sets of visual tokens. The final hidden states represent the appropriateness of a contextualized edit rather than the aesthetics of an isolated image. The output side includes a text decoder and an MLP score head, supporting textual level learning and fine-grained continuous regression, respectively. The three scores share the input representation but retain distinct evaluation meanings, instead of collapsing all supervision into one scalar at the outset.
The paper calls this adaptive decoding: establish stable textual outputs before predicting scores from the hidden states. The prose and Figure 4 establish that textual level learning precedes fine-grained scoring, but do not specify a reproducible inference-time stability-detection algorithm. It should therefore not be interpreted as an additional self-checking loop that repeatedly generates text for every evaluation. At test time, three-dimensional scores for each candidate suffice for comparison, without requesting new human annotations. Candidates share the source image and prompt, preserving the editing context on which their preferences depend. Although absolute scoring and within-group ranking share a model, they require different evaluation metrics; neither capability should substitute for validation of the other.
3. Staged joint training: learn quality levels before combining score calibration and within-group order
Training first divides the observed range between minimum and maximum scores uniformly into 5 intervals labeled bad, poor, fair, good, and excellent. This textual learning stage uses cross-entropy to establish coarse quality semantics in a multimodal model already capable of language understanding. The result is not intended to remain a five-class evaluator: subsequent continuous supervision restores finer distinctions. Pointwise learning trains the score head on human-scored examples using mean squared error, encouraging predictions to approximate absolute quality values. This gives outputs from different source images and prompts a common numerical scale, instead of only identifying the best candidate within each group. Regression alone may still misorder closely scored images, motivating preference supervision derived from the rankings.
Pairwise learning encourages preferred images to score above rejected images, first prioritizing larger rank gaps and later handling subtler differences between nearby ranks. This coarse-to-fine ordering reduces the initial burden of learning difficult candidate comparisons. The exponential loss in cached Equation (5) has missing symbols, so a guessed expression is not presented as the authors' exact formula. The confirmed mechanism penalizes incorrect preference order, rather than training the model only to emit a discrete answer such as choosing the left image. The complete training uses both pointwise and pairwise supervision: one constrains absolute score positions, while the other improves relative discrimination under the same editing conditions. Table 4 supports their complementarity, but the main text does not clearly specify joint loss weights or every stage-switching condition, leaving these details to be verified for reproduction.
4. Reward-guided editing: convert evaluation into trainable chosen/rejected pairs
For a source image and editing prompt, the method generates a candidate group and asks EditHF for visual quality, editing alignment, and attribute preservation scores. Equation (2) on page 7 combines these dimensions with a weighted geometric mean, assigning slightly more weight to instruction fulfillment. Here, \(s_v\), \(s_e\), and \(s_p\) denote the three dimensions, and \(s_{all}\) ranks candidates within their group.
The highest-scoring image becomes chosen and the lowest becomes rejected, but the pair is discarded when even the best candidate has low quality. This avoids treating the least unsuccessful output in a failed group as a desirable positive example. Section 6.5 further describes retaining the top half of scored pairs within each task to control preference data quality. Self-DPO samples candidates from Qwen-Image-Edit using 20 different random seeds per input, producing 18K preference pairs. Global-DPO instead draws candidates from the multiple editing models in EditHF-1M, producing 22K preference pairs and exposing the editor to outputs unavailable through its own sampling. The two approaches change the source of preference candidates, not the definition of the evaluator.
Editor refinement uses direct preference optimization (DPO), freezing the VAE while updating the Transformer and retaining the original pretrained editor as a reference model. Training compares flow prediction errors on noisy chosen and rejected latents, favoring preferred outputs relative to the reference. The logarithm, signs, and term connections in cached Equation (6) are incomplete, so the exact flow-DPO objective is not reconstructed here. The reward primarily constructs offline preference pairs; the paper's reinforcement learning terminology should not be taken to imply online policy-gradient training. After refinement, ordinary editing inference still accepts a source image and prompt and does not require human rankings for every generation. The paper checks post-training gains with additional human ratings, so its evidence is not limited to improving the same evaluator used for optimization.
Loss & Training¶
The evaluator uses a 5:1:1 training, validation, and test split, and test images carry both rankings and scores. This passage does not explicitly state whether splitting is performed by source-image group, so complete exclusion of shared-source leakage should not be assumed. Both vision and language models use LoRA, while the projector and score head provide cross-modal connections and numerical outputs. The implementation uses PyTorch and reports training on 2 NVIDIA RTX A6000 GPUs; this statement describes the EditHF experimental setup. Evaluator supervision and editor DPO are separate learning processes, and the latter's 18K/22K preference pairs must not be confused with the evaluator's full training data. The project URL comes from the abstract, which states that data and code will be released; release status and unreported hyperparameters are not inferred from that statement.
Key Experimental Results¶
Main Results¶
Table 2 on page 11 reports four metric types on the EditHF-1M test set, all higher-is-better. SRCCglobal measures global rank correlation with human scores, PLCCglobal measures linear correlation, SRCCgroup averages rank correlations within shared-source, shared-prompt groups, and Acc measures pairwise comparison accuracy. The table below compares EditHF with the fine-tuned InternVL3 baseline, rather than presenting a zero-shot baseline as equally supervised.
| Method | Dimension | SRCCglobal | PLCCglobal | SRCCgroup | Acc |
|---|---|---|---|---|---|
| InternVL3, fine-tuned | Visual quality | 0.8044 | 0.7852 | 0.8284 | 0.8817 |
| EditHF | Visual quality | 0.8548 | 0.8422 | 0.9223 | 0.9152 |
| InternVL3, fine-tuned | Editing alignment | 0.8248 | 0.8156 | 0.8540 | 0.8498 |
| EditHF | Editing alignment | 0.8488 | 0.8348 | 0.9319 | 0.9298 |
| InternVL3, fine-tuned | Attribute preservation | 0.8120 | 0.8297 | 0.9137 | 0.8883 |
| EditHF | Attribute preservation | 0.8475 | 0.8442 | 0.9251 | 0.9197 |
Ablation Study¶
Table 4 on page 13 holds InternVL3.5 and the MLP head fixed; the excerpt below isolates pointwise versus pairwise supervision for editing alignment. These configurations belong to the same ablation and should not be assumed to use the identical training recipe as the separate fine-tuned baselines in Table 2.
| Config | SRCCglobal | SRCCgroup | Acc |
|---|---|---|---|
| Pointwise only | 0.7806 | 0.7964 | 0.7825 |
| Pairwise only | 0.8213 | 0.8894 | 0.8762 |
| Pointwise and pairwise combined | 0.8488 | 0.9319 | 0.9298 |
The combined configuration raises editing-alignment Acc by 0.0536 over pairwise-only training, or 5.36 percentage points, not a 5.36% relative improvement. Pairwise learning helps within-group ranking, while additional pointwise supervision improves both ranking and global correlation, showing that absolute scoring is more than an auxiliary output format.
Key Findings¶
Table 6 on page 14 compares the original Qwen-Image-Edit with both post-training approaches; only human overall scores are reproduced below, excluding the table's separate EditHF automatic scores. The benchmarks use the human scoring protocol from Section 3.3, and the last column contains the paper's reported relative Global-DPO improvement over the original model.
| Benchmark | Original model | Self-DPO | Global-DPO | Global-DPO relative gain |
|---|---|---|---|---|
| EditHF-1M | 75.39 | 78.36 | 84.26 | +11.77% |
| AROURA-Bench (source table spelling) | 83.21 | 84.28 | 85.46 | +2.70% |
| ImagenHub | 75.64 | 76.55 | 79.12 | +4.60% |
| EBench-18K | 73.25 | 73.80 | 74.18 | +1.27% |
| EditScore-Bench | 70.17 | 72.18 | 75.23 | +7.21% |
| EditReward-Bench | 80.15 | 81.17 | 82.60 | +3.06% |
Moving from 75.39 to 84.26 on EditHF-1M adds 8.87 score points, corresponding to the reported +11.77% relative gain, not an increase of 11.77 score points. Global-DPO exceeds Self-DPO on all these human scores, but their datasets contain 22K and 18K pairs, respectively, so the entire gain cannot be attributed to cross-model diversity. Cross-benchmark results in Table 5 on page 13 additionally report GenAI-Bench accuracy of 0.6792 and ImagenHub SRCC of 0.4526; these different metrics must not be combined into a single win rate. That table averages across dimensions where applicable and excludes some models trained on the target benchmark; this supports transfer, but overlap in data sources still requires separate examination.
Highlights & Insights¶
- Separate scores for visual quality, instruction fulfillment, and preservation distinguish different failure types. A single aggregate score can conceal successful editing that damages identity.
- Group rankings expand into large-scale preference supervision, while fewer absolute ratings supply a global scale. Their value lies in complementary information, not in treating every expanded pair as independent human labor.
- Multi-model candidates expose an editor to high-quality outputs beyond its own stochastic samples. Human evaluation gains support refinement more strongly than reward-score increases alone, although they do not eliminate reward bias.
Limitations & Future Work¶
- Section 6.1 states split proportions without adequately specifying source-group isolation, while Section 3.2 uses EBench-18K data. Cross-dataset results do not automatically establish generalization with fully disjoint source images.
- Global-DPO changes both candidate sources and the number of preference pairs, without a matched-size, matched-filtering comparison. This limits a reader's causal interpretation rather than constituting a result established by the authors.
- Expanded preference pairs are correlated, and removing inconsistent annotations may suppress subjective diversity. Future work could retain disagreement distributions and report calibration across tasks or annotator populations.
- Cached Equations (1), (5), and (6) have extraction corruption, and the main text incompletely specifies loss weights and filtering thresholds. Exact reproduction requires a clear manuscript, supplementary material, and implementation; this note does not guess those details.
Related Work & Insights¶
- Compared with EditScore and EditReward: these works already explore editing reward models; this paper further emphasizes broader task and model coverage and joint learning from absolute scores and relative preferences. Introducing reward-based editor refinement for the first time is not its contribution.
- Compared with EBench-18K / LMM4Edit: the paper expands smaller opinion-score evaluation to million-scale outputs and large-scale group rankings, and tests post-training benefits. Shared data sources nevertheless require a more explicit transfer-evaluation protocol.
- Compared with Q-align: textual quality levels exploit the language model's existing semantics; the continuous MLP score head and pairwise learning then extend level understanding into fine-grained editing comparisons.
- Transferable insight: video editing or local restoration could similarly evaluate instruction fulfillment separately from preservation constraints; directly reusing this scorer would still require experiments on temporal consistency and the new task distribution.
Rating¶
- Novelty: 4/5. The main advance is large-scale dual-feedback data and unified evaluation training, not a new preference optimization algorithm.
- Experimental Thoroughness: 4/5. Correlation analysis, training ablations, cross-benchmark evaluation, and human post-training validation are present, but data isolation and matched-size controls remain incomplete.
- Writing Quality: 3/5. The overall approach is clear, but implementation details are limited and corrupted loss equations in the current extraction hinder precise understanding for reproduction.
- Value: 4/5. Useful for editing evaluation and preference data construction, with practical reuse still dependent on data and implementation availability.