MirrorPPR: Exemplar-Based Portrait Photo Retouching¶

Conference: ECCV2026
arXiv: 2606.29308
Code: None (Project page https://sjtu-deng-lab.github.io/MirrorPPR might be released later)
Area: Image Generation
Keywords: Portrait Retouching, Exemplar-Based Editing, Diffusion Transformer, LoRA Fine-Tuning, Self-Augmentation

TL;DR¶

MirrorPPR proposes an exemplar-guided portrait structural retouching task. Through a dedicated editing operation extractor (MAE + R-Former), a connector, a DiT dual-stream backbone (Qwen-Image-Edit-2511), and a LoRA adaptation pipeline, combined with a 47-million-pair dataset generated by a self-augmentation data strategy, it achieves precise cross-identity transfer of fine structural retouching operations for the first time.

Background & Motivation¶

Text-guided image editing has achieved remarkable progress in recent years. New-generation models from InstructPix2Pix to Qwen-Image-Edit and FLUX.2 can handle diverse editing requests. However, when the task is applied to portrait structural retouching, text descriptions reveal inherent limitations. Structural retouching involves extremely fine-grained operations—narrowing the nose wings by 2mm, raising the mouth corners by 3 degrees, or slightly narrowing the jawline—where natural language cannot quantitatively specify the precise spatial scale, direction, and magnitude of these geometric deformations. When a user says "enlarge the eyes a bit," the model does not know whether "a bit" means 5% or 20%. Similarly, "slim the face" could mean reducing the jaw or compressing the cheekbones. This ambiguity often leads text-guided models to either under-edit or over-edit, especially causing anatomically exaggerated "over-retouching" on facial features, which severely damages the portrait's identity.

Exemplar-based editing bypasses the expressive bottlenecks of text. Users intuitively demonstrate their desired operations via a "before-after" exemplar pair, from which the model infers the editing intention and applies it to another query image. While successful in style transfer and object replacement, transferring this to portrait structural retouching presents two new challenges. First, structural retouching operations are extremely subtle—shrinking lips, minor nose wing reduction, or narrowing the nasal bridge—where the visual differences are far smaller than those in style transfer or object editing. Existing exemplar-guided models (e.g., EditTransfer, RelationAdapter) lack the sensitivity to perceive such fine-grained differences, often misinterpreting the operations as image fusion or face swapping. Second, training data faces a severe bottleneck: constructing cross-identity quadruplets (where the same operations are applied to two different faces) is extremely difficult. Due to variations in capture scale, pose, occlusion, and facial proportions, it is virtually impossible to strictly align the same operations across different faces, leading to ambiguous supervision signals and hindering the model from learning a consistent "operation \(\rightarrow\) transfer" mapping.

This work addresses both bottlenecks simultaneously. At the modeling level, a dedicated editing operation extractor is designed. It leverages a frozen MAE to preserve local geometric details, combined with a learnable-query Transformer (R-Former) to extract subtle editing representations from the exemplar pair. These are then injected into a pre-trained Diffusion Transformer (DiT) via a connector. At the data level, a self-augmentation data generation strategy is proposed. Given only a single exemplar pair, applying the exact same set of random spatial augmentations (rotation, cropping, flipping) to both the before and after images automatically synthesizes query pairs. This ensures perfectly aligned operations while breaking pixel-level correspondences, successfully bypassing the cross-identity alignment challenge. Based on this strategy, the MirrorPPR47M dataset containing 47 million pairs is constructed, and a progressive curriculum learning scheme is designed, transitioning from simulated warping to professional retouching. Core Idea: The fine-grained portrait structural retouching is decoupled into two phases: "operation extraction" and "operation transfer." The former employs MAE + a learnable-query Transformer to perceive subtle editing differences, while the latter utilizes a connector to inject these into a frozen DiT for zero-shot transfer. Concurrently, synchronous spatial self-augmentation is used to construct perfectly aligned training quadruplets, eliminating the cross-identity misalignment bottleneck.

Method¶

Overall Architecture¶

The overall pipeline of MirrorPPR consists of two sequential phases: the editing operation extractor perceives the structural retouching differences in the exemplar pair (before image Xs, after image Xt) and outputs editing representations; this representation is mapped by a connector and injected as a condition into the pre-trained DiT dual-stream backbone (Qwen-Image-Edit-2511), which is end-to-end fine-tuned with LoRA, ultimately transferring the same retouching operations to a new query image Xq. The entire framework is trained using a progressive two-stage process: "pre-train the extractor first \(\rightarrow\) then joint fine-tuning".

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Exemplar Pair (Xs, Xt)"] --> B["MAE (Frozen)"]
    B --> C["R-Former<br/>Learnable Query Interaction"]
    C --> D["Editing Representation H_edit"]
    D --> E["Connector<br/>Feature Mapping"]
    E --> F["Editing Condition c_edit"]

    G["Query Image Xq"] --> H["Qwen2.5-VL<br/>(Frozen)"]
    G --> I["VAE Encoder<br/>(Frozen)"]
    H --> J["Visual Semantics c_img"]
    I --> K["Latent Space z_cond"]

    F --> L["DiT Dual-Stream Backbone<br/>(Frozen + LoRA)"]
    J --> L
    K --> L

    L --> M["Retouching Result Y^q"]

Key Designs¶

1. R-Former Learnable Query: Extracting Subtle Editing Representations from Exemplar Pairs

The visual differences of structural retouching operations are extremely subtle, which conventional encoders struggle to perceive. Inspired by Moto, this work designs a Retouching Operation Extractor, consisting of a frozen MAE and a trainable R-Former. The masked pre-training of MAE naturally preserves local spatial structure and geometric information, making it suitable for capturing pixel-level fine-grained differences. The exemplar pair Xs and Xt are separately fed into the frozen MAE to obtain patch-level features.

The actual role of "extracting differences" is undertaken by R-Former, a standard ViT architecture but with a set of internal learnable query tokens (default to 8) concatenated along the sequence dimension. These query tokens "read out" representations H_edit that represent the retouching operations from the MAE features through self-attention interaction mechanisms. Intuitively, query tokens repeatedly interact with image features across multiple self-attention layers, gradually learning "which regions changed, and in which direction/by how much," while ignoring unchanged identity information. To ensure that H_edit indeed contains accurate retouching operation information, the extractor is supervised during the pre-training stage with an auxiliary reconstruction task: an MLP compresses H_edit into a compact editing embedding e_edit, which is added to each patch token of the query image, and a lightweight ViT decoder is used to reconstruct the retouched result, optimized by MSE + LPIPS loss. After pre-training, the auxiliary decoder is discarded, leaving the purely semantic editing representation H_edit.

2. Connector + LoRA Dual-Stream Injection: Efficiently Adapting Editing Conditions to Pre-trained Diffusion Models

Once H_edit is obtained, it needs to be injected into the image editing diffusion model to drive retouching operations. This work chooses Qwen-Image-Edit-2511 as the backbone, which employs a dual-stream DiT architecture (Qwen2.5-VL to extract high-level visual semantics + a diffusion stream to perform denoising in the VAE latent space) and receives textual instructions as conditions. Since this task is exemplar-driven and does not require text, the visual editing conditions must be integrated into this dual-stream system.

The key design is a Connector employing an Enc-Proj structure: first, a Transformer encoder aligns H_edit with the instruction conditioning space of the backbone (dimension 768 \(\rightarrow\) 3584), and then a linear projection layer maps it to the input dimension of the DiT blocks. After the connector, c_edit is input to the DiT dual-stream architecture for diffusion denoising, along with the visual features c_img extracted by Qwen2.5-VL and the image latent z_cond encoded by VAE.

To efficiently leverage the strong priors of the pre-trained DiT, the original parameters of the backbone are fully frozen, and only trainable LoRA modules (rank=32) are inserted into its attention blocks. During the joint fine-tuning stage, only the parameters of R-Former, the connector, and the LoRA modules are updated, optimized using the flow matching loss (velocity MSE). This strategy of freezing the backbone while using lightweight adaptation retains the pre-trained editing capabilities of large models while avoiding catastrophic forgetting and the high computational cost of full-parameter fine-tuning.

3. Self-Augmentation and Progressive Curriculum Learning: Eliminating Cross-Identity Alignment Issues and Progressively Mastering Professional Operations

When constructing training quadruplets (exemplar pair Xs \(\rightarrow\) Xt + query pair Xq \(\rightarrow\) Yq) for exemplar-guided retouching, the most straightforward approach is to use cross-identity data, where the same set of operations are applied to two different faces. However, in practice, due to variations in pose, occlusion, and shot scale, the pixel-level implementation of the same retouching operation cannot be strictly matched across different face shapes—narrowing the nose wings manifests completely differently on a frontal face versus a 45-degree profile. This misalignment of operations results in conflicting supervision signals, preventing the model from learning stable operation transfers.

A seemingly simple alternative is to use the exemplar pair itself as the query pair (Self w/o Aug): setting Xq=Xs, Yq=Xt. Although this avoids misalignment, it introduces severe shortcut learning—the model only needs to copy pixel coordinate differences without demonstrating any understanding of the "operation semantics," causing it to completely fail during cross-identity testing. The core data strategy of this work is self-augmentation: applying the exact same random spatial augmentation \(A\) (rotation, horizontal flipping, dynamic cropping) to both exemplar images Xs and Xt to construct the query pair Xq=A(Xs), Yq=A(Xt). Since the augmentations are synchronized across both images, the operation offsets remain strictly consistent. Furthermore, because the augmentations break absolute pixel coordinate correspondences, the model cannot take the "coordinate copying" shortcut and must learn the generalization capability of "extracting operation semantics from the exemplar pair \(\rightarrow\) applying operations based on the spatial layout of the query image itself." Self-augmentation training not only matches or even exceeds the performance of ideal cross-identity training (which actually performs worse on professional data due to misalignment), but also converges much faster.

Based on the self-augmentation strategy, a large amount of retouching data is required. This work constructs the MirrorPPR47M dataset: the simulated subset uses landmark-guided local warping (LLW) algorithms on 30,000 faces from FFHQ to generate 800,000 pairs of obvious deformation retouching across 8 categories of operations; the professional subset uses professional commercial retouching APIs on 3,789 4K-8K portraits from PPR10K to generate 46.64 million pairs of fine retouching across 27 categories of operations. These two subsets form a progressive curriculum—first pre-training the extractor on simulated data to perceive basic structural deformations, then transitioning to professional data to adapt to real-world fine retouching, and finally performing joint fine-tuning of the entire framework using only professional data.

An Illustrative Example¶

Let us assume an exemplar pair: Xs has a natural mouth shape, and Xt is the retouched result with the lips narrowed by 2mm and the nose wings minimized. The input query image Xq is a new face. The flow is as follows: Xs and Xt are respectively passed through the frozen MAE to extract patch features. The 8 query tokens of R-Former "read out" two operation differences via self-attention interaction, producing H_edit (an 8×768 feature). The connector maps H_edit to the DiT condition c_edit (dimension 3584). At the same time, Xq is sent to Qwen2.5-VL to extract semantic features c_img and to the VAE encoder to obtain the latent space representation z_cond. These three sets of conditions (c_edit, c_img, z_cond) are input into the DiT dual-stream backbone, generating the final result Y^q after 40 denoising steps—resulting in the query face having a smaller mouth and narrowed nose wings, while unchanged areas such as the background and skin textures are completely preserved. During training, self-augmentation randomly rotates Xs and Xt synchronously by 10 degrees and crops them to different regions so that the mouth shape of Xq=A(Xs) is no longer pixel-aligned with Xs, yet the relative offset of the mouth narrowing (direction and magnitude of pixel displacement) remains consistent between Xs \(\rightarrow\) Xt and Xq \(\rightarrow\) Yq, forcing the model to learn the operation semantics of "mouth narrowing" indeed instead of copying coordinates.

Loss & Training¶

The extractor pre-training stage uses MSE + LPIPS reconstruction loss: \(\mathcal{L}_{pre} = \|\hat{Y}_q - Y_q\|_2^2 + \lambda \mathcal{L}_{lpips}(\hat{Y}_q, Y_q)\), where \(\lambda=1.0\). The joint fine-tuning stage employs the flow matching loss: \(\mathcal{L}_{flow} = \mathbb{E}_{\mathbf{z}_1,\mathbf{z}_0,t}[\|\mathbf{v}_\theta([\mathbf{z}_t,\mathbf{z}_{cond}],t,\mathbf{c}_{img},\mathbf{c}_{edit}) - \mathbf{v}_t\|_2^2]\), where \(\mathbf{z}_t = t\mathbf{z}_0 + (1-t)\mathbf{z}_1\) is the linearly interpolated noisy latent, and \(\mathbf{v}_t = \mathbf{z}_0 - \mathbf{z}_1\) is the target velocity. The pre-training learning rate is 1e-4 (cosine decayed to 5e-5), and the joint fine-tuning learning rate is 1e-5 (constant), optimized using AdamW.

Key Experimental Results¶

Main Results¶

Evaluated on two cross-identity test benchmarks: SimFace-100 (simulated retouching, 8 operation categories × 12 faces) and ProPortrait-500 (professional retouching, 27 operation categories × 40 faces). Baseline models include multi-reference image editing (Qwen-Image-Edit-2511, FLUX.2-dev, Nano Banana 2, Seedream 4.5), exemplar-guided editing (ICEdit-LoRA, RelationAdapter, EditTransfer), and text-guided editing (same set of models with text prompts).

Benchmark	Metric	MirrorPPR	Best Baseline	Gain
SimFace-100	PSNR ↑	32.25	25.80 (text Qwen)	+6.45
SimFace-100	SSIM ↑	0.909	0.862 (text Qwen)	+0.047
SimFace-100	LPIPS ↓	0.186	0.239 (text Nano Banana 2)	-0.053
SimFace-100	Face Similarity ↑	0.937	0.601 (text Nano Banana 2)	+0.336
ProPortrait-500	PSNR ↑	32.65	27.45 (text Nano Banana 2)	+5.20
ProPortrait-500	SSIM ↑	0.927	0.904 (text Nano Banana 2)	+0.023
ProPortrait-500	LPIPS ↓	0.200	0.183 (text Nano Banana 2)	+0.017
ProPortrait-500	Face Similarity ↑	0.960	0.811 (multi-reference Nano Banana 2)	+0.149

Notably, multi-reference and exemplar-guided baselines fail almost completely (with PSNRs generally below 18), indicating their inability to perceive such subtle structural edits. Text-guided baselines perform relatively well in pixel-level reconstruction (due to preserving the global structure) but obtain significantly lower Face Similarity. For instance, Nano Banana 2 achieves an LPIPS of 0.183 on ProPortrait-500, but its Face Similarity is only 0.667, indicating severe disruption to identity features. MirrorPPR leads across all metrics, particularly outperforming other models on Face Similarity by a large margin (0.960 vs. the next best 0.811), demonstrating that it perfectly preserves identity features while accurately transferring retouching operations.

Ablation Study¶

Configuration	PSNR (SimFace-100)	Face Similarity	Description
Self w/o Aug	28.63	0.680	No spatial augmentation, severe shortcut learning
Cross-Identity	32.08	0.937	Ideal cross-identity alignment (controllable on simulated data)
Self-Augmentation (Ours)	32.25	0.937	Bypasses cross-identity alignment, performance matches ideal alignment
Cross-Identity (Professional Data)	30.52	0.916	Severe cross-identity misalignment on professional data
Self-Augmentation (Professional Data)	32.65	0.960	Self-augmentation yields significant gains on professional data

Core ablation conclusions: (1) Self w/o Aug without augmentation significantly degrades due to coordinate-copying shortcuts; (2) On simulated data (where operations are controllable), self-augmentation matches ideal cross-identity alignment; (3) On professional data (with complex operations and diverse poses), self-augmentation significantly outperforms cross-identity training (PSNR +2.13, Face Similarity +0.044) because cross-identity alignment is harder for fine-grained operations; (4) Self-augmentation converges faster.

Key Findings¶

Vector additivity in editing representation space: After performing instance-level mean-centering on the extracted editing embedding e_edit, the editing direction vectors for different operations can be directly added to synthesize composite editing effects (e.g., "narrow eye corner direction vector + thicken lip direction vector" \(\rightarrow\) composite operation). The PSNR of the vector addition reaches 30.85, surpassing all baseline models (excluding MirrorPPR itself). This indicates that the learned editing space is decoupled and composition-friendly.
79.0% User Preference: In a user study on ProPortrait-500 with 100 samples, participants blindly selected MirrorPPR-Pro with a 79.0% preference rate (compared to only 20.6% for the runner-up text-guided Nano Banana 2), indicating its advantages are clearly perceived by real users.
Insensitivity to query token count: The performance of 8 / 64 / 256 query tokens is almost identical (PSNR difference < 0.1), implying that 8 is sufficient; LoRA rank 32 achieves the best balance between speed and performance.

Highlights & Insights¶

Decoupling operation extraction from operation transfer is a core design wisdom: the extraction stage (R-Former) focuses on perceiving "differences," while the transfer stage (DiT + LoRA) focuses on "applying differences." Both sub-problems are addressed with the most appropriate techniques without interfering with each other, introducing a clearer inductive bias compared to end-to-end direct generation.
The elegance of the self-augmentation strategy lies in its two-step "break and establish" approach: first, using homogeneous construction ensures operation alignment (breaking cross-identity misalignment), and then using synchronous spatial augmentation eliminates absolute coordinate correspondences (breaking shortcut learning). These two "breaks" yield a pure semantic learning signal for the operation. This is an elegant insight from a data engineering perspective.
Vector additivity of editing embeddings validates spatial decoupling: after removing identity features via mean-centering, the remaining editing direction vectors can be manipulated independently and combined freely, essentially establishing an "editing operation algebra" in the latent space. This property holds strong practical value for future composite retouching (such as predicting the sequential effects of A and then B).
Using frozen MAE + R-Former instead of an end-to-end trained encoder: the geometric representation advantages of MAE (previously demonstrated by Xie et al. 2023) are cleverly repurposed for the "difference perception" xstask, avoiding the need to train a large encoder from scratch. This is an efficient and data-economical approach to transfer learning.

Limitations & Future Work¶

Dependency on frozen MAE and DiT backbone: The performance upper bound of the framework is constrained by the capability of MAE to perceive extremely fine-grained operations and the quality of the editing prior of Qwen-Image-Edit-2511. If the backbone is replaced (e.g., with a stronger future DiT model), the connector and LoRA will need to be re-adapted.
Lack of evaluation on 3D deformation operations: Currently, all 27 professional operations are 2D deformations (MLS deformation guided by facial landmarks). This might be insufficient for 3D structural retouching involving depth or perspective changes (such as adjusting nose bridge height under varying 3D perspective projections). Future work could consider incorporating 3D facial priors.
Dataset limited to single portraits: The self-augmentation strategy relies on single-person bounding box cropping and does not currently cover multi-portrait scenes (such as adjusting individuals in a group photo).
Limited types of spatial augmentations in self-augmentation: Current augmentations (rotation, horizontal flip, dynamic cropping) are primarily affine and do not account for non-uniform deformations (e.g., fisheye distortion) or illumination changes. Introducing richer augmentations may further enhance generalization.

vs. Moto: Moto proposes using MAE + learnable queries to extract motion representations. MirrorPPR transfers this core idea to the domain of portrait structural retouching. The key difference is that MirrorPPR injects the extracted editing representations into DiT via a connector, rather than simply decoding them into a target image, thus achieving cross-identity transfer.
vs. EditTransfer / RelationAdapter: These exemplar-guided methods are effective for tasks like style transfer or object editing, but they lack mechanisms to perceive extremely fine-grained differences. MirrorPPR's MAE + R-Former extractor remains effective at micro-operation scales where these methods are blind.
vs. Text-guided editing (InstructPix2Pix, FLUX.2-dev): Text-guided methods are naturally inadequate when handling quantitative instructions like "enlarge eyes by 10%". MirrorPPR bypasses linguistic ambiguity using visual exemplars, which represents a fundamental paradigm shift—not improving text understanding, but directly switching to a different communication channel.
vs. MetaQuery: The connector design borrows the Enc-Proj structure from MetaQuery to align conditioning spaces of different modalities. However, MirrorPPR's condition source is a visual editing representation rather than text, marking the first time this design is applied to retouching operation injection.

Rating¶

Novelty: ⭐⭐⭐⭐⭐ Proposes the entirely new task of "exemplar-guided portrait structural retouching" and provides a complete solution from model to data. The self-augmentation strategy is ingenious and highly generalizable.
Experimental Thoroughness: ⭐⭐⭐⭐⭐ Conducts comprehensive validation across three baseline types (multi-reference / exemplar-guided / text-guided), 8 models, and two benchmarks, alongside ablation studies, latent space analysis, and user studies.
Writing Quality: ⭐⭐⭐⭐⭐ The chain of motivation is clear (textual limitations \(\rightarrow\) exemplar paradigm \(\rightarrow\) micro-edit perception challenge \(\rightarrow\) cross-identity misalignment \(\rightarrow\) two-step solution), and the narration of key designs is smooth and free of redundancy.
Value: ⭐⭐⭐⭐⭐ Resolves the issue of precise control over "where to retouch and how much" in real retouching scenarios. The team has committed to open-sourcing the model, data, and datasets, presenting extremely high practical value.