FED-Bench: A Cross-Granular Benchmark for Disentangled Evaluation of Facial Expression Editing¶

Conference: ECCV2026
Paper: ECCV
Code: https://github.com/hiixfj/FED-Bench
Area: Image Generation
Keywords: Facial Expression Editing, Image Editing Benchmark, Disentangled Evaluation, Cross-Granular, Data Construction

TL;DR¶

FED-Bench builds 747 (source image / editing instruction / ground-truth) triplets with a cascaded pipeline — generate candidates, verify expressions via three-polarity recognition, rank by dual-anchor fidelity preservation, then human re-check — and proposes FED-Score, which disentangles evaluation into three orthogonal dimensions (Fidelity, Alignment, Relative Expression Gain) combined multiplicatively, suppressing both lazy editing and overfit editing across 18 editing models.

Background & Motivation¶

Text-guided image editing has matured rapidly with diffusion models: InstructPix2Pix established the mainstream paradigm of training conditional diffusion models on synthetic paired data, AnyEdit added task-aware routing and mixture-of-experts to cover more edit types, OmniGen used a single Transformer backbone for joint text-image encoding, and Step1X-Edit improved complex instruction following and background consistency with a dual-stream bridging mechanism. Facial expression editing is one of the most demanding applications of this capability — it is not merely "make this face a bit happier," but changing only the subtle movement of facial muscles while strictly preserving the subject's identity and background. Existing editing benchmarks (ImgEdit, EditBench, I2EBench) target general scenarios: they lack high-quality facial samples and never design instructions for the subtle movements of facial muscles. More critically, most provide only a source image and a text instruction with no ground-truth reference image — without a pixel-level anchor, evaluation can only stay at a coarse-grained or subjective level.

Beyond the data, the metrics themselves carry a systemic bias. When the weighting leans toward fidelity metrics (e.g., DINO Score), the protocol rewards lazy editing: a model that simply outputs the source image earns an exceptionally high score. When it leans toward semantic alignment metrics (e.g., CLIP Score), the protocol rewards overfit editing: the model generates exaggerated, distorted expressions to cater to the prompt at the cost of identity. This bias does not just distort rankings; it misdirects the design of future models — a model that only copies its input outranking one that actually edits is simply a wrong signal.

The key enabler that makes this tractable now is that editing models can already synthesize sufficiently realistic candidate targets, while MLLMs can both analyze fine-grained visual differences (articulating how facial muscles moved between two images) and act as automatic judges. That makes "synthetic candidates + multi-stage automatic filtering + human verification" a scalable route to near-ground-truth references, without waiting for genuinely paired in-the-wild expression data. The paper accordingly makes two complementary moves: a cascaded pipeline that filters candidates down to a trustworthy ground truth (polarity-reduced expression verification, then dual-anchor identity/background ranking, then human finalization plus dense-instruction generation), and a rejection of "tuning weights on a single metric" in favor of decomposing editing quality into non-substitutable dimensions that must all hold at once. Core idea: filter generated candidates into 747 ground-truth triplets through a scalable cascaded pipeline, and score models with a multiplicative FED-Score over fidelity × alignment × relative expression gain, so that a collapse in any one dimension can never be compensated by another.

Method¶

Overall Architecture¶

The paper produces two things: the benchmark itself — FED-Bench, 747 triplets plus an extensible 20k+ training set — and the evaluation protocol FED-Score. On the construction side, the input is in-the-wild facial images (drawn from SFEW 2.0 and DFEW, two movie-clip-derived expression datasets) and the output is (source image / editing instruction / ground-truth target) triplets. The flow is: clean the source images, use an editing model to generate candidates for the other six expressions of each source, then pass them through three successive funnels — expression verification, fidelity ranking, human verification — collapsing the candidate set to a single ground truth, and finally have an MLLM generate a dense instruction by comparing the source with the ground truth. On the evaluation side, 18 editing models are run over the full test set, each scored once under simple instructions and once under dense instructions, then aggregated into FED-Score over the three dimensions. The pipeline never hand-draws a ground truth, so it scales by swapping parts: replace the source pool with 20k+ in-the-wild faces from RAF and DFEW and the generator with Qwen-Image-Edit-2511, and the released 20k+ training set is what comes out.

Where the "cross-granular" part lands: the paper never defines the term item by item, but the protocol's composition shows it spans scales in three places at once. First, metric scale — background consistency sits at the pixel level (RMSE outside the face mask), identity preservation at the feature level (ArcFace embedding cosine), expression-change magnitude at the perceptual level (LPIPS on the face crop), and semantics is delegated to MLLM scoring, so four levels from pixels to semantics are combined. Second, signal source — rule-based computation (ID, BG, REG) is mixed with model-based perception (PQ, SC, GTA); the paper's own phrasing is that the protocol "deeply integrates rule-based computations with the model-based perceptual capabilities of MLLMs." Third, instruction granularity — every triplet carries both a template-style simple instruction and a dense instruction, and evaluation runs under both, exposing how much a model's ability shifts with control precision.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Source screening<br/>SFEW 2.0 + DFEW"] --> B["Candidate expression verification<br/>three-polarity reduction + multi-model voting"]
    B --> C["Ground-truth finalization<br/>dual-anchor ranking + human check + dense instruction"]
    C --> D["FED-Bench<br/>747 triplets"]
    D --> E["FED-Score disentanglement<br/>fidelity × alignment × relative gain"]
    E -->|third dimension| F["Relative Expression Gain<br/>Gaussian penalty anchored at GT"]
    E --> G["Rankings of 18 models<br/>fidelity-alignment trade-off"]

Key Designs¶

1. Candidate expression verification: collapse seven expressions into three polarities, then make automatic filtering usable with multi-model voting

The candidate targets are synthesized, so the pipeline must verify that they actually manifest the requested expression; otherwise the whole benchmark degenerates into samples whose instruction says "make it happy" while the image still shows the original expression. But expert FER models all but fail on in-the-wild faces (movie clips, occlusion, extreme lighting): BEiT, Luxand, and DeepFace reach only 35.7%, 39.8%, and 36.5% on the seven-class task, and even the strongest MLLM, Gemini-2.5-Pro, reaches only 64.93%. Filtering strictly at that accuracy discards many samples that are in fact valid, while loosening the threshold lets wrong expressions leak into the ground truth.

The paper's answer is to abandon fine-grained classes and judge polarity only: regrouping the seven expressions into Positive (happy), Neutral, and Negative (the remaining five) immediately lifts the peak accuracy from 64.93% to 81.23%. A multi-model voting ensemble is then layered on top: with five models (Gemini-2.5-Pro, GLM-4.6V, Kimi-K2.5, ERNIE-5.0, and Step-3; the coarse-grained variant swaps Step-3 for Doubao-seed-1.6-vision), coarse-grained peak accuracy reaches 84.87% with 82.27% average. This works because the requirement is matched to model capability: filtering only needs to answer "did the expression really change, and did it move to the positive or the negative side," which is exactly the granularity of polarity judgment, whereas seven-way discrimination lies beyond what current MLLMs do reliably on in-the-wild data; voting then averages out single-model misjudgments. The cost is that this layer only guarantees the polarity is right — within a polarity a surprise can still be taken for happy — and that residual error is absorbed by the next layer's fidelity ranking and human verification.

2. Ground-truth finalization: dual-anchor fidelity ranking, human verification, and a dense-instruction return path

Each source image has six candidates (one per remaining expression), and after expression verification several usually survive, so the pipeline must pick the one that most looks like "the same face, only the expression changed." It uses two complementary anchors for what should stay unchanged: ArcFace extracts identity features and computes the cosine similarity between source and candidate (identity preservation), and RMSE is computed over pixels outside the face mask (background consistency). After normalization, the two are fused with weights into an overall preservation score, and only the top two candidates proceed to the human stage. That human stage is not ceremonial — automatic filtering cannot see subtle facial dynamics or microscopic artifacts — so an MLLM first writes a detailed expression caption for each candidate as an objective reference, and three annotators then independently inspect the candidate pairs for expression naturalness, instruction alignment, and identity/background preservation, with majority voting settling the single ground truth.

A return path follows finalization: the source image and the ground truth are fed together to an MLLM as a visual difference analyzer (how the facial muscles moved, how eye curvature changed, whether the mouth opened or closed), producing a highly descriptive dense editing instruction for each triplet that sits alongside the template instruction ("change the expression from {src_emotion} to {trg_emotion}") used during screening. This dense instruction serves two purposes: a finer control signal for generative models, and a harder tier of linguistic input for evaluation — it raises "instruction following" from "can you understand happy" to "can you realize a smile with upturned mouth corners, slightly narrowed eyes, and visible upper teeth."

3. FED-Score disentanglement: split evaluation into three orthogonal dimensions, then require all of them at once via a product

A single metric can always be gamed: judging by fidelity alone rewards lazy editing (copy the source image and ID and BG are near-perfect), while judging by alignment alone rewards overfit editing (paint an exaggerated, distorted expression to push semantic similarity up). Rather than hunting for a balance point by tuning weights, the paper decomposes "a good edit" into three non-substitutable things and requires them simultaneously. The three dimensions are fidelity \(\mathcal{S}_{\text{fid}}\), alignment \(\mathcal{S}_{\text{align}}\), and relative expression gain \(\mathcal{S}_{\text{reg}}\) (the mechanism of the third is the next design).

Fidelity averages three components equally: identity preservation (ID) is the ArcFace embedding cosine between source and generated image; background consistency (BG) is the RMSE over pixels outside the face mask, passed through a normalization function so that higher is better; perceptual quality (PQ) is scored by an MLLM for artifacts and degradation. Alignment averages two: semantic consistency (SC) has an MLLM judge how well the generated image matches the input instruction, while ground-truth-based expression alignment (GTA) hands the generated image and the benchmark's ground truth to an MLLM to compare expression similarity — a tier of supervision that only exists once a ground truth exists, and is far more reliable than text-only alignment because it bypasses the inherent imprecision of mapping language onto expressions. The three dimensions combine by product rather than weighted sum:

\[\text{FED-Score}=\mathcal{S}_{\text{fid}}\times\mathcal{S}_{\text{align}}\times\mathcal{S}_{\text{reg}}\]

The product means any dimension near zero drags the total near zero: a model cannot compensate a collapse in one dimension with a high score in another, which is precisely why lazy and overfit editors score highly under weighted sums. The ablation in Tab. 3d supports this "all are necessary" claim — removing any single dimension lowers agreement with human judgment.

4. Relative Expression Gain: normalize the edit magnitude by the ground truth, penalizing both lazy and overfit editing

Even with fidelity and alignment in place, the lazy-editing loophole remains: outputting the source gives perfect ID and BG, and alignment scoring is insensitive to "only changed a tiny bit" — for "make this face happy," a barely-there smile still passes semantic matching. The paper therefore measures how much actually changed, and that quantity is only meaningful relative to the ground truth. It computes the LPIPS perceptual distance from source to generated image on the face crop, then normalizes it by the source-to-ground-truth distance to obtain the ratio \(r\): \(r=1\) means the magnitude of change matches the ground truth exactly. So that this dimension does not itself become "the more change the better," the final score is a Gaussian penalty centered at 1.0:

\[\mathcal{S}_{\text{reg}}=\exp\!\left(-\frac{(r-1)^2}{2\sigma^2}\right),\qquad r=\frac{\text{LPIPS}_{\text{face}}(I_{\text{src}},I_{\text{trg}})}{\text{LPIPS}_{\text{face}}(I_{\text{src}},I_{\text{gt}})}\]

\(\sigma\) controls the tolerance width and is set to 0.5 in the experiments. Lazy editing with \(r\ll1\) and overfit editing with \(r\gg1\) are thus penalized by the same score, and only a change magnitude matching the ground truth earns the full 1.0 — one metric closing off two opposite failure modes. The validation in Tab. 3b shows this ratio-based metric tracks human judgment better than alternatives: LPIPS-Ratio_F reaches 0.7386, above LPIPS_F,GT (0.7045), DINO_F,GT (0.6818), and the other ratio-based metric, RPM-Modify_F (0.6154), while identity-oriented measures such as ArcFace_F,GT (0.5455) and CLIP-I_F,GT (0.5909) fall to near chance, indicating their feature spaces simply do not capture expression-change magnitude. The authors also mark the boundary: REG measures the perceptual magnitude of change in the face region, a proxy for expression-change intensity rather than a semantic action-unit measure, and the LPIPS ratio will also absorb some texture and lighting variation; the supplement validates it against AU-based measures.

A Worked Example¶

Take one neutral source image that passed cleaning (numbers below are illustrative). In the candidate stage, Nano Banana generates one candidate per remaining expression, six in total. In the verification stage, the three-polarity reduction judges one candidate positive, three negative, and two neutral — the two neutrals are discarded (one of them because the model did not change the expression at all) — and after five-model voting, four candidates survive. Fidelity ranking scores them with weighted ArcFace cosine and background RMSE and keeps the top two. Three annotators then re-check and majority-vote the happy candidate as the ground truth, eliminating the other. After finalization, an MLLM compares source and ground truth and writes a dense instruction such as "mouth corners curve up into a smile, eyes narrow slightly, upper teeth visible." At evaluation time, suppose a model's face-region LPIPS distance from the source is only half that of the ground truth: then \(r\approx0.5\) and REG scores \(\exp(-0.5)\approx0.61\) — even with near-perfect ID and BG, the total is dragged down by this one dimension.

Loss & Training¶

The evaluation side of this paper is a pure inference-time protocol and involves no training; the only training-side action validates the scalability of the data-construction paradigm: Qwen-Image-Edit-2511 is used to build training pairs from 20k+ in-the-wild faces in RAF and DFEW with the same recipe, which then fine-tunes FLUX-Kontext-Dev into FLUX-Kontext-FED. The paper states only that fine-tuning substantially improves expression accuracy and fidelity, without reporting learning rate or step counts.

Key Experimental Results¶

Main Results¶

Metric reliability is validated with a 2AFC human study: 2,760 pairs of editing results are sampled at random (two different models' outputs for the same source image and instruction), three annotators each pick the better image from three perspectives (identity preservation, expression change magnitude, overall quality), majority voting forms the human consensus, and the agreement rate between each metric's preference and that consensus is reported.

Dimension	Metric	Agreement with humans (Acc.)
Identity preservation	ArcFace_F	0.6606
Identity preservation	DINO_F	0.6596
Identity preservation	LPIPS_F	0.6528
Identity preservation	CLIP-T_F	0.5420
Expression change magnitude	LPIPS-Ratio_F (this paper's REG)	0.7386
Expression change magnitude	LPIPS_F,GT	0.7045
Expression change magnitude	DINO_F,GT	0.6818
Expression change magnitude	RPM-Modify_F	0.6154
Expression change magnitude	ArcFace_F,GT	0.5455
Overall quality	FED-Score (this paper)	0.7700
Overall quality	CLIP-I_GT	0.6850
Overall quality	CLIP-I	0.6325
Overall quality	CLIP-T	0.4480

Rankings of 18 editing models on the full FED-Bench test set (Dense = dense instructions, Simple = simple instructions; higher ID is better, REG is optimal at 1.0):

Method	Dense Score	Dense ID	Dense REG	Dense GTA	Simple Score	Simple REG
Qwen-image-edit-plus	.469	.58	1.18	5.7	.492	1.13
SeedDream 4.0	.379	.62	1.37	4.3	.413	1.26
FLUX.2 Pro	.377	.58	1.37	4.0	.400	1.37
Qwen-image-edit	.337	.45	1.37	4.2	.343	1.37
FLUX-Kontext-FED (fine-tuned here)	.332	.68	0.95	3.4	.325	0.96
FLUX-Kontext-Pro	.327	.52	0.99	3.5	.227	0.97
FLUX-Kontext-Max	.320	.50	1.07	3.5	.259	1.03
Qwen-image-edit-2511	.317	.46	1.43	3.4	.361	1.42
Step1X v1p2	.303	.56	1.31	3.0	.333	1.41
FLUX-Kontext-Dev	.243	.72	0.67	3.0	.120	0.35
SeedEdit 3.0	.203	.49	1.55	1.9	.239	1.56
UniWorld-v2	.201	.37	1.45	2.5	.110	1.60
DreamOmni2	.168	.81	0.45	2.6	.049	1.40
OmniGen2	.155	.56	1.52	2.6	.122	1.61
FLUX-Kontext-Fill	.155	.11	1.56	1.7	.096	1.52
Step1X	.127	.33	1.72	1.3	.149	1.67
Bagel	.115	.42	1.57	1.8	.163	1.29
InstructPix2Pix	.001	.08	2.56	0.0	.004	2.42

Ablation Study¶

Ablation of the individual FED-Score dimensions (again measured as agreement with human judgment):

Config	Acc.	Note
FED-Score (full)	0.7700	full protocol
w/o REG	0.7577	removes relative expression gain, −0.0123
w/o Rule	0.7422	removes all rule-based metrics (ID / BG / REG), −0.0278
w/o Fidelity	0.7379	removes the fidelity dimension, −0.0321
w/o Alignment	0.7279	removes the alignment dimension, −0.0421
w/o Model	0.7202	removes MLLM scoring (PQ / SC / GTA), −0.0498

The two levels of the data pipeline's expression-verification layer (polarity reduction and voting ensembles):

Config	7-class Fine Best / Avg	3-polarity Coarse Best / Avg
Expert FER (BEiT / Luxand / DeepFace)	35.7 / 39.8 / 36.5	–
Best single model (Gemini-2.5-Pro)	64.93	81.23
3-model voting	66.13 / 59.94	84.47 / 81.04
5-model voting	66.67 / 61.75	84.87 / 82.27
7-model voting	66.00 / 62.78	84.61 / 82.88
9-model voting	63.99 / 63.25	83.40 / 82.90

Key Findings¶

No model achieves both high fidelity and high alignment, which is exactly what FED-Score is built to expose. FLUX-Kontext-Dev and DreamOmni2 achieve the highest identity preservation (ID .72 and .81) but only 0.67 and 0.45 REG — classic lazy editing, preserving identity by barely touching the expression. InstructPix2Pix is the textbook overfit editor in the opposite direction: REG as high as 2.56 while perceptual quality PQ collapses to 0.0, leaving its total near zero (.001). FLUX-Kontext-Fill is a third failure mode: the best background consistency (BG 5.1) and perceptual quality (PQ 9.9) but the worst identity preservation (ID .11) — a high-quality image of a different person. Judged by any single metric, lazy editors would be ranked at the top on ID or BG alone.
Fine-grained instruction following is the current bottleneck. Alignment metrics (SC, GTA) vary the most between instruction granularities: FLUX-Kontext-Pro's SC nearly halves from Dense to Simple, suggesting it leans disproportionately on the detailed guidance in dense instructions, while FLUX-Kontext-Dev's lazy tendency worsens under simple instructions (REG falling from 0.67 to 0.35). The top-3 ranking is stable across both settings (Qwen-image-edit-plus, SeedDream 4.0, FLUX.2 Pro), indicating that these models have genuine general editing proficiency rather than a specialty at one instruction granularity.
The released data does help: fine-tuning FLUX-Kontext-Dev on the 20k+ synthetic training pairs yields FLUX-Kontext-FED, which rises from 10th to 5th under dense instructions (.243 → .332) and from 14th to 7th under simple ones (.120 → .325), with REG moving from 0.67/0.35 to 0.95/0.96 — right around the ideal 1.0, showing the lazy-editing habit was corrected by the data.
The MLLM scoring components are the least dispensable in the ablation: removing PQ/SC/GTA drops agreement to 0.7202 (a 0.0498 fall), more than removing any rule-based dimension, showing that perceptual quality and semantic consistency cannot be replaced by rule-based computation; conversely, w/o Rule (0.7422) is also clearly below the full protocol, showing that objective identity and background measurements retain independent value even next to a strong vision-language model. Dropping REG alone hurts least (0.7577), as expected — it targets expression-change magnitude specifically, a dimension less relevant in identity-focused and quality-focused comparison pairs.
Text-image similarity metrics are unreliable for this task: CLIP-T_F reaches only 0.5420 on the identity dimension, and CLIP-T only 0.4480 on overall quality, below the 0.5 random baseline, while every "compared against the ground truth" variant consistently beats its source-target counterpart, confirming the value of a GT reference for evaluation.

Highlights & Insights¶

Reducing granularity to sidestep a model's capability ceiling is a transferable filtering design. When the automatic judge is not accurate enough at fine granularity (64.93% over seven classes), filtering strictly costs both recall and precision; dropping the judgment target to a granularity the model handles reliably (84.87% over three polarities) and then voting reaches a workable point between "clean enough" and "large enough." Any pipeline that needs LLM/MLLM data QC can copy this move: first ask how fine the judgment really needs to be, then ask whether the model is stable at that granularity.
REG's use of a ratio plus Gaussian penalty, rather than a distance, is the cleverest step in the protocol. Using the perceptual distance to the ground truth directly would encourage the model to approach that one image and could still be gamed by exaggeration; isolating the score at "ratio = 1" instead funnels two opposite failures — lazy (ratio < 1) and overfit (ratio > 1) — into the same penalty, and makes the metric independent of any absolute scale.
Multiplicative aggregation writes "balance" into the protocol itself. A weighted sum can always be offset by one high-scoring dimension, whereas a product lets any collapse veto the total, so the leaderboard stops rewarding models that push one aspect to an extreme. This aggregation transfers directly to any multi-dimensional quality assessment (quality/consistency/motion magnitude for video generation, or correctness/efficiency/safety for agents).
The dense-instruction return path is cheap added value: the ground truth already exists, so having an MLLM compare it with the source and write a detailed instruction yields both a finer control signal and a harder evaluation input at essentially no extra cost. This "harvest a second supervision signal from existing annotation" pattern is well worth reusing in dataset construction.

Limitations & Future Work¶

The authors explicitly acknowledge it: the reference images are pseudo ground truth — synthesized by an editing model rather than captured from the same subject — so each reflects one plausible target rather than a unique answer. The robustness analysis in the supplement shows model rankings stay stable when the reference generator and MLLM judge are swapped, but the underlying ambiguity (one person performing one expression admits many valid renderings) is not removed; collecting truly paired expression data remains future work.
REG is a perceptual proxy that absorbs texture and lighting variation and is not equivalent to an AU-level semantic measure. A result that altered the lighting while matching the ground truth's change magnitude could therefore be judged reasonable; the paper only provides an AU comparison in the supplement and does not quantify this deviation in the main text.
The benchmark is small (747 triplets) and its source images all come from SFEW 2.0 and DFEW, two movie-derived datasets, so the face distribution skews toward film scenes and particular demographics; selfies, surveillance, and animation styles are uncovered. Evaluation covers only the seven basic expressions — compound expressions (e.g., pleasant surprise) and intensity gradations (a smile versus a grin) are out of scope, even though the latter are exactly the ambiguous cases users write most often.
The three perceptual scoring components (PQ, SC, GTA) are all delegated to a single model, Gemini-2.5-Pro, which carries single-judge bias risk; whether stability holds under a different judge is discussed only in the supplement, with no cross-judge validation in the main experiments.
One improvement direction: replace LPIPS in the REG ratio with an estimate of facial action-unit intensity, keeping the ratio form while bringing the proxy closer to expression semantics. Another is to give each triplet several valid ground truths and move evaluation from comparison against one reference to comparison against a distribution of references, addressing the multi-solution nature of pseudo ground truth at the root.

vs ImgEdit / EditBench / I2EBench: they target general editing (adding or removing objects, colour changes, global style transfer) and provide only a source image and an instruction; FED-Bench is specific to facial expressions and supplies high-quality in-the-wild sources, instructions designed for subtle facial movement, and crucially a ground-truth reference. The consequence is direct: without a ground truth, only reference-free metrics such as CLIP or DINO are available, and those are precisely the metrics that breed lazy and overfit editing.
vs MagicBrush / HQ-Edit / AnyEdit (data side): these datasets cover general edit operations via manual annotation or synthetic pipelines at much larger scale, but none filters for the fine-grained expression task. FED-Bench makes the opposite trade-off — only 747 triplets, each passed through expression verification, dual-anchor identity/background ranking, and three human reviewers — favouring "small but trustworthy."
vs VIEScore (metric side): the MLLM scoring framework here (PQ / SC / GTA) is directly inspired by VIEScore in using an MLLM as judge; the difference is that FED-Bench mixes MLLM scoring with rule-based metrics (ArcFace identity, background RMSE, LPIPS ratio) and uses a product to force the dimensions to hold jointly, rather than handing all the weight to a model judge that may carry its own systemic preferences.
vs MagicFace / LaTo (method side): these achieve fine-grained expression editing with structured geometric conditions such as action-unit deltas or facial landmarks, and work well, but they rely on supervised labels or geometric priors and cannot consume natural-language instructions directly. This paper proposes no new editing model; it contributes evaluation and training data, and the FLUX-Kontext-FED fine-tuning result shows the data can in turn improve a general instruction-editing model.

Rating¶

Novelty: ⭐⭐⭐⭐ [the first fine-grained facial-expression editing benchmark with ground-truth references, with REG using a ratio plus Gaussian penalty to close off lazy and overfit editing at once; the problem is precisely located and the solution is unflashy but on target]
Experimental Thoroughness: ⭐⭐⭐⭐ [18 models × two instruction granularities, a 2,760-pair 2AFC human study, a full dimensional ablation, and a two-level ablation inside the data pipeline; the gaps are a single MLLM judge for all perceptual components and the REG-versus-AU comparison buried in the supplement]
Writing Quality: ⭐⭐⭐⭐ [the lazy-versus-overfit opposition in the motivation is stated crisply, and each protocol dimension's definition cross-validates with its ablation; OCR quality of the tables is mediocre and a few formulas are corrupted in the source layout]
Value: ⭐⭐⭐⭐ [data, evaluation protocol, and a 20k training set are all released; diagnosing the lazy/overfit failure modes is directly useful to any work on editing evaluation, with the main discounts being the pseudo ground truth and the small size of 747]