Towards In-Context Tone Style Transfer with A Large-Scale Triplet Dataset¶

Conference: ECCV 2026
Paper: ECCV Official
Code: Available online (Project page)
Area: Image Generation
Keywords: Tone Style Transfer, Photo Retouching, In-Context Generation, Triplet Dataset, Reward Feedback Learning

TL;DR¶

Addressing the lack of high-quality paired triplet data and semantic color misalignments in tone style transfer, this paper introduces TST100K—a 100K content-reference-stylized dataset curated via a two-stage tone scorer and aesthetic filtering—and proposes ICTone, a flow-matching DiT framework that treats retouching as in-context visual generation paired with scorer-driven reward feedback learning.

Background & Motivation¶

Tone style transfer for photographic retouching is a fundamental problem in computational photography and image editing. It aims to adapt tone-related photographic attributes—including global color distribution, luminance, contrast, and saturation—from a reference exemplar to a target content image, while strictly maintaining the geometric content and structural fidelity of the input. However, the field has long been bottlenecked by the fundamental absence of ground-truth stylized targets for content-reference image pairs. Consequently, established approaches such as Neural Preset and CAP-VSTNet have been forced to rely on self-supervised formulations or proxy style losses, which inherently constrain the model's capacity to handle challenging cross-domain tone mapping.

The principal obstacle to constructing high-quality triplet supervision lies in the absence of an objective, perceptual metric for tone style similarity. Manual retouching by professional colorists yields reliable pairs but remains prohibitively expensive and unscalable; conversely, blindly applying predefined photography presets across arbitrary images creates severe perceptual inconsistencies, as distinct image semantics react disparately to identical color curves. At the architecture level, traditional methods rely on decoupled feature extraction—encoding content and style independently before fusing them in a decoder. This structural decoupling leads to semantic information loss, frequently producing unnatural artifacts such as background color bleeding onto human faces and inconsistent regional tonality.

To overcome these intertwined bottlenecks, this work tackles both data curation and architectural formulation. On the data side, the authors design a closed-loop curation pipeline driven by a dedicated, two-stage tone similarity scorer alongside aesthetic quality verification. On the modeling side, they reframe tone transfer as an in-context generative task powered by modern diffusion transformers. Core idea: build the 100K paired triplet dataset TST100K via a contrastively trained and preference-tuned tone style scorer, and formulate tone transfer as an in-context generation task within a flow-matching DiT (ICTone) boosted by scorer-guided reward feedback learning.

Method¶

Overall Architecture¶

ICTone aims to model the conditional distribution $p_\theta(I_t \mid I_c, I_r)$, where given an unretouched content photograph $I_c$ and a stylistic reference image $I_r$, the model generates a stylized output $I_t$ that embodies the tonal characteristics of $I_r$ while preserving the structural composition of $I_c$. The overall pipeline operates across two tightly linked phases: first, constructing the large-scale TST100K dataset and TST2K benchmark using a two-stage trained tone style scorer and aesthetic filtering; second, training the ICTone framework by concatenating $I_c$ and $I_r$ into a single context canvas for masked inpainting via a flow-matching Diffusion Transformer (DiT), refined at later stages via reward feedback learning from the frozen tone scorer.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Data Collection & Preprocessing<br/>White Balance + Filter Removal + 3K Presets"] --> B["Two-Stage Tone Style Scorer<br/>Weakly-Supervised Contrastive + Preference Triplet Tuning"]
    B --> C["High-Fidelity Triplet Dataset Construction<br/>Tone Consistency (>=0.8) + Aesthetic Non-Degradation"]
    C --> D["In-Context Generation Framework ICTone<br/>Spatial Canvas (Ic, Ir) + Masked Flow-Matching DiT"]
    D --> E["Tone Reward Feedback Learning<br/>Scorer Backpropagation + Perceptual Style Alignment"]
    E --> F["Output High-Fidelity Retouched Image It"]

Key Designs¶

1. Two-Stage Tone Style Scorer: Resolving the Lack of Objective Tone Similarity Metrics To establish a principled, scalable metric for tone style similarity without semantic confounding, the authors adopt a CLIP ViT-B/16 backbone with a normalized projection head to map images into a specialized tone embedding space. In the first stage, the scorer is trained via weakly-supervised contrastive learning on over 3,000 preset-generated categories: pairs generated by identical presets serve as positives $\mathcal{P}(i)$, while those from distinct presets serve as negatives. Image blurring is applied as augmentation to suppress high-frequency structural details and force the model to focus on color distributions and global illumination patterns, optimized via supervised contrastive loss: $$ \mathcal{L}{\mathrm{Con}} = - \sum}^{N} \frac{1}{|\mathcal{P}(i)|} \sum_{p \in \mathcal{P}(i)} \log \frac{\exp(\mathbf{zi \cdot \mathbf{z}_p / \tau)}{\sum} \exp(\mathbf{zi \cdot \mathbf{z}_a / \tau)} $$ Because single presets occasionally produce divergent perceptual effects on heterogeneous scenes, the second stage introduces preference learning. Hard negative pairs where multiple discriminators (VGG, ResNet, ViT) disagree are curated into 20K human-ranked annotations. Evaluators rank candidate harmony and tone consistency, supervised by a margin triplet loss ($m=0.3$): $$ \mathcal{L}_n) + m\right} $$ where }} = \max\left{0,\, d(\mathbf{z}_a, \mathbf{z}_p) - d(\mathbf{z}_a, \mathbf{z$d(\mathbf{u}, \mathbf{v}) = 1 - \frac{\langle \mathbf{u}, \mathbf{v} \rangle}{\|\mathbf{u}\|_2 \|\mathbf{v}\|_2}$ is the cosine distance, aligning the scorer's metric space with human subjective perception.

2. High-Fidelity Triplet Dataset Construction: Dual Verification Balancing Quality and Scale Leveraging the trained MTS-Scorer and an aesthetic assessment model (AesCLIP), the authors build TST100K (100,000 triplets) and TST2K (a 2,000-triplet benchmark). Raw photographs from PPR10K and MIT-Adobe FiveK undergo automated white balance correction and filter removal to eliminate preexisting color casts before applying curated photography presets. To eliminate low-quality and visually inconsistent pairings, two complementary filtering gates are enforced: an aesthetic non-degradation constraint (retaining only stylized images whose aesthetic score meets or exceeds the original image) and a tone consistency constraint (discarding reference-stylized pairs with tone embedding cosine similarity below 0.8). Online content augmentations—including random desaturation, exposure reduction, and LUT perturbations—are incorporated during model training to expand generalization across realistic degraded inputs.

3. In-Context Generation Framework: Joint Semantic-Aware Transfer via Flow-Matching DiT To circumvent semantic information loss caused by separate content and style encoders in prior works, ICTone reformulates tone transfer as an in-context visual generation problem. Built on the FLUX.1 Fill DiT backbone, the model concatenates content image $I_c$ and reference image $I_r$ along the spatial dimension within a single input canvas. The region designated for the stylized output is initialized with random noise and treated as a masked inpainting target. Through global self-attention across the combined canvas, the transformer naturally establishes semantic correspondences between content regions and reference style elements—without requiring explicit content-style disentanglement modules.

4. Tone Reward Feedback Learning: Enforcing Perceptual Style Alignment While flow-matching provides a stable generation baseline, generative models may suffer from color mode collapse or under-stylization in extreme lighting conditions. Inspired by ReFL, ICTone incorporates direct reward feedback learning from the frozen MTS-Scorer. For any generated output $\hat{I}_i = I_\theta(I_c^i, I_r^i)$, the scorer computes its tone similarity against the reference image $I_r^i$ and backpropagates a reward-guided gradient: $$ \mathcal{L}{\mathrm{tone}} = \mathbb{E}}}\big[\phi\big(\mathcal{M{\mathrm{TS\text{-}Scorer}}(I_r^i,\, I\theta(I_c^i, I_r^i))\big)\big] $$ This reward loss penalizes tonal drift and guides the diffusion trajectory toward sharper contrast, higher perceptual harmony, and accurate highlight/shadow color reproduction.

Loss & Training¶

The overall training objective combines the continuous-time flow-matching loss with the tone reward feedback loss: $$ \mathcal{L} = \mathcal{L}{\mathrm{FM}} + \lambda $$ where }} \mathcal{L}_{\mathrm{tone}$\mathcal{L}_{\mathrm{FM}} = \mathbb{E}_{\boldsymbol{s}_0, t, \epsilon} \|\boldsymbol{v}_\theta - \boldsymbol{v}_t\|^2$, velocity target $v_t = \frac{d\alpha_t}{dt}s_0 + \frac{d\sigma_t}{dt}\epsilon$ with $\alpha_t = 1-t, \sigma_t = t$. To ensure training stability, $\lambda_{\mathrm{tone}}$ is scheduled as 0 before training step $S$ and 1 thereafter. ICTone is fine-tuned on FLUX.1 Fill using LoRA on 4 NVIDIA A100 GPUs with the AdamW optimizer (learning rate $1 \times 10^{-4}$, weight decay $1 \times 10^{-3}$) for a total of 50,000 iterations.

Key Experimental Results¶

Main Results¶

Quantitative evaluations are conducted on the newly proposed TST2K benchmark (2,000 curated real-world triplets) and the public PST50 dataset (50 pairs, testing zero-shot generalization). Evaluation metrics include Content Preservation (CP, SSIM computed on LDC edge maps), Color Difference ($\Delta E$, CIEDE2000), Deep Color Difference (CD), and Aesthetic Quality (Aes).

Dataset	Method	CP ↑	$\Delta E$ ↓	CD ↓	Aes ↑
TST2K	WCT2 [54]	0.3952	15.13	4.991	0.7136
	PhotoNAS [3]	0.6721	19.57	6.183	0.6347
	MKL [35]	0.7511	12.67	4.694	0.7363
	PhotoWCT [26]	0.5852	21.35	7.021	0.6788
	DeepPreset [15]	0.7728	8.205	4.108	0.7567
	ModFlows [21]	0.6908	12.43	4.945	0.7500
	IPST [29]	0.7364	9.648	4.326	0.7081
	RLPixTuner [48]	0.6815	25.61	7.514	0.6354
	SA-LUT [13]	0.7082	11.10	4.929	0.6507
	CAP-VSTNet [46]	0.7013	11.87	4.665	0.7175
	CAP-VSTNet* [46] (retrained)	0.7665	7.359	3.663	0.7801
	Neural Preset [18]	0.7480	10.51	4.713	0.7551
	Neural Preset* [18] (retrained)	0.7707	7.886	3.860	0.7808
	CSGO [51]	0.3440	19.00	7.166	0.7209
	Nano Banana2 [14]	0.5968	12.78	4.947	0.7885
	ICTone (Ours)	0.8644	5.776	2.634	0.7904
	Ground Truth	1.000	0.000	0.000	0.7978
PST50	WCT2 [54]	0.6482	16.34	5.574	0.2381
	MKL [35]	0.7618	16.12	5.241	0.2742
	SA-LUT [13]	0.7697	11.25	4.483	0.2048
	CAP-VSTNet* [46] (retrained)	0.7545	14.45	4.937	0.2779
	Neural Preset* [18] (retrained)	0.6914	17.81	6.157	0.1714
	ICTone (Ours)	0.7902	12.81	4.448	0.2820
	Ground Truth	1.000	0.000	0.000	0.3305

In the user study conducted with 20 participants comparing 7 leading methods across 120 pairwise comparisons per method, ICTone achieved an average ranking of 1.00 (rank 1 across all participants), noticeably outperforming Neural Preset (2.70), MKL (3.60), SA-LUT (4.05), and CAP-VSTNet (4.35).

Ablation Study¶

1. Impact of High-Quality Dataset Filtering (on TST2K, without reward learning): | Configuration | CP ↑ | $\Delta E$ ↓ | CD ↓ | AesScore ↑ | Note | |---|---|---|---|---|---| | w/o filtering | 0.7950 | 9.859 | 3.674 | 0.7727 | Unchecked noisy presets impair training | | w/ filtering | 0.8567 | 6.035 | 2.737 | 0.7834 | $\Delta E$ drops by ~39% with dual filtering |

2. Retrieval and Preference Accuracy of Tone Scorer on IFFI Dataset: | Scorer Variant | Recall@1 ↑ | Recall@2 ↑ | Recall@5 ↑ | PAcc (Preference Acc) ↑ | |---|---|---|---|---| | VGG Gram [17] | 34.00% | 47.75% | 67.44% | 58.21% | | Neural Disc [18] | 34.63% | 48.19% | 69.94% | 65.12% | | CSD [42] | 63.13% | 73.75% | 87.50% | 70.89% | | TS-WCL (Stage 1 only) | 93.19% | 97.44% | 99.38% | 74.59% | | TS-WCL-PL (Stage 1 + Stage 2) | 97.50% | 99.56% | 100.0% | 82.67% |

3. Component Breakdown of Tone Reward Learning: | WCL (Contrastive) | PL (Preference) | CP ↑ | $\Delta E$ ↓ | CD ↓ | AesScore ↑ | |:---:|:---:|---|---|---|---| | - | - | 0.8567 | 6.035 | 2.737 | 0.7834 | | ✓ | - | 0.8572 | 5.795 | 2.765 | 0.7871 | | ✓ | ✓ | 0.8644 | 5.776 | 2.634 | 0.7904 |

Key Findings¶

Large-scale paired supervision fundamentally unlocks model performance: Retraining CAP-VSTNet and Neural Preset on TST100K reduces $\Delta E$ from 11.87 to 7.359 and 10.51 to 7.886 respectively, with simultaneous aesthetic improvements, proving that previous performance ceilings stemmed directly from unsupervised proxy objectives.
Human preference fine-tuning aligns the metric with perceptual judgment: Moving from TS-WCL to TS-WCL-PL improves retrieval Recall@1 from 93.19% to 97.50% and preference accuracy from 74.59% to 82.67%, ensuring that downstream reward learning optimizes for human-pleasing color harmony rather than mathematical artifacts.
In-context spatial reasoning prevents color bleeding: Qualitative analysis reveals that while prior methods contaminate foreground skin tones with background reference colors, ICTone leverages DiT global self-attention to maintain correct semantic tone correspondences (e.g., natural portrait skin tones vs. background atmosphere).

Highlights & Insights¶

Dual utilization of the tone scorer across data curation and model optimization: Rather than treating data cleaning and model training as disconnected stages, the paper deploys the tone scorer both offline as a hard filtering gate ($\ge 0.8$) and online as a soft reward signal for policy gradient refinement.
In-context visual reasoning replaces ad-hoc feature disentanglement: Instead of designing specialized modules (e.g., AdaIN, WCT, or 4D LUTs) to prevent structural distortion, the paper proves that DiT models can naturally disentangle semantic content and color style purely through in-context concatenation.
Pragmatic and clean data synthesis pipeline: Combining white balance normalization and filter stripping before applying 3,000 professional presets provides a scalable recipe for generating diverse photographic paired datasets without licensing issues.

Limitations & Future Work¶

Stylistic bias in automated aesthetic scorers: Relying on AesCLIP for data filtering inherently introduces inductive biases toward high-contrast, saturated images, potentially discarding subdued or low-key moody photographic presets.
Computational overhead of spatial image concatenation: Concatenating high-resolution content and reference images within the DiT input canvas causes quadratic self-attention memory growth, limiting inference speed for interactive real-time mobile deployment.
Future directions: Investigating cross-attention adapters or decoupled conditioning mechanisms to accelerate inference; developing user-adaptive aesthetic scorers that accommodate personalized artistic preferences.

vs Neural Preset [18]: Neural Preset estimates compact retouching parameters via self-supervision on single images. While lightweight and fast, it struggles with complex cross-scene lighting shifts. ICTone provides true triplet supervision and leverages generative priors for superior semantic-aware color mapping.
vs SA-LUT [13]: SA-LUT introduces spatially adaptive 4D LUTs for ultra-fast photorealistic transfer. While efficient, it lacks deep conceptual semantics, occasionally transferring irrelevant ambient colors to semantic foregrounds; ICTone demonstrates superior deep color fidelity (CD 2.634 vs 4.929 on TST2K).
vs In-Context Generation [24, 57]: Prior in-context vision models primarily focus on object insertion or instructional edits. This work successfully extends in-context learning to fine-grained, continuous photographic tone transfer.

Rating¶

Novelty: ⭐⭐⭐⭐☆ [First closed-loop framework combining a dedicated tone scorer, 100K triplet dataset, and in-context diffusion retouching]
Experimental Thoroughness: ⭐⭐⭐⭐⭐ [Extensive cross-benchmark evaluations, thorough ablations on filtering and reward stages, plus comprehensive user studies]
Writing Quality: ⭐⭐⭐⭐⭐ [Exemplary clarity in pipeline illustration, mathematically sound formulations, and consistent notation]
Value: ⭐⭐⭐⭐⭐ [Provides a definitive paired benchmark and foundational dataset that resolves a decade-long data scarcity bottleneck in tone style transfer]