Skip to content

Enlightening Photographic Style Transfer with a Self-Supervised Photographic Embedding

Conference: ECCV 2026
Paper: ECCV Official
Project: https://petal-pku.pages.dev/
Area: Image Generation
Keywords: Photographic Style Transfer, Self-Supervised Learning, Image Retouching, Lookup Table (LUT), Continuous Style Representation

TL;DR

PETAL proposes a dedicated photographic style embedding network trained via self-supervised photographic style augmentation and CIE-Lab histogram guidance, conditioning a lightweight adaptive 5D neural LUT to achieve high-fidelity photographic style transfer without test-time optimization.

Background & Motivation

Style transfer is a classic problem in computer vision, yet prior efforts have predominantly focused on artistic stylization—such as painterly brushstrokes and abstract texture synthesis—or crude global lightness shifts. Real-world photography retouching is fundamentally different: professional photographers meticulously adjust continuous tonal attributes (exposure, color temperature, tone curves, contrast) and apply smooth spatial adjustments such as vignetting, all while strictly preserving underlying scene content and high-frequency textures. Existing representation models rely heavily on discrete text supervision or semantic classification categories (e.g., CLIP, DINO), which cannot capture fine-grained, continuous shifts in exposure or tint. Applying such representations to style transfer inevitably leads to mismatched tone features and distorted textures.

On the other hand, lookup table (LUT)-based retouching methods inherently preserve original textures thanks to their spatially independent pixel mappings. However, when handling complex or stylized photographic references, existing LUT formulations frequently fail to capture subtle contrast levels or cause unnatural color shifts due to poorly aligned reference style descriptions. Diffusion-based retouching and deep feature statistics alternative approaches either distort image geometry, cause color bleed, or suffer from prohibitive inference latency.

The central bottleneck therefore lies in learning an embedding space that is sharply sensitive to continuous photographic retouching while disentangled from semantic content. The core idea is to decouple semantics from tone by leveraging contrary photographic style augmentations as hard negative pairs, learning a continuous photographic embedding via CIE-Lab histogram cross-attention, and conditioning a lightweight adaptive 5D neural LUT on style embedding deltas to enable zero-test-time-optimization style transfer.

Method

Overall Architecture

PETAL consists of two decoupled stages: (1) a Photographic Style Embedding Network (\(\mathcal{M}\)) that extracts a compact, content-invariant, tone-sensitive embedding from any reference photo, and (2) a lightweight Adaptive 5D Neural LUT (\(\mathcal{E}, \mathcal{D}\)) that uses embedding deltas between reference and content images to modulate pixel colors and spatial coordinates via AdaIN-like affine transformations.

The entire framework requires no test-time optimization, generating stylized high-resolution outputs in a single forward pass.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Input: Content and Style Reference Images"] --> B["CIE-Lab Histogram & ViT Dual Encoding<br/>Decouple luminance/chroma & extract global tonal distributions"]
    B --> C["Self-Supervised Contrastive Learning<br/>Local jitter positives & contrary augmentation hard negatives"]
    C --> D["Adaptive 5D Neural LUT Modulation<br/>Embedding delta conditioned AdaIN affine transformation"]
    D --> E["Output: High-Fidelity Retouched Image"]

Key Designs

1. CIE-Lab Histogram & ViT Dual Encoding: Decoupling Luminance-Chrominance and Capturing Global Distributions

Standard RGB color representations entangle luminance and color tone, making it difficult for neural networks to separate semantic objects from exposure and color temperature. PETAL first transforms input images into the CIE-Lab color space to explicitly isolate the luminance channel from chromaticity. The main backbone is a Vision Transformer (ViT) that processes patch tokens alongside a global [CLS] token. In parallel, the network computes a 1D luminance histogram \(h^L \in \mathbb{R}^H\) and a 2D joint chromaticity histogram \(h^{ab} \in \mathbb{R}^{H \times H}\) using differentiable inverse-quadratic kernels:

\[ h^L(i) = \sum_{p=1}^N \mathcal{K}(I^L(p), b_i), \quad h^{ab}(i, j) = \sum_{p=1}^N \mathcal{K}(I^a(p), b_i)\mathcal{K}(I^b(p), b_j) \]

where \(\mathcal{K}\) is a smooth kernel function and \(b_i, b_j\) are bin centers. These histograms are processed by dedicated lightweight encoders and concatenated into \(H_{Lab}\), which is then injected into ViT patch representations via cross-attention. Finally, the [CLS] token is concatenated with adaptively pooled patch tokens \(\mathcal{P}(V^{\text{p}})\) and projected to form the final embedding \(z = h([V^{\text{cls}}; \mathcal{P}(V^{\text{p}})])\), jointly capturing global statistical distributions and spatial tonal nuances.

2. Self-Supervised Contrastive Learning: Contrary Augmentation as Hard Negatives

The continuous nature of photography makes discrete text labels unsuitable for supervision. Moreover, naive patch contrastive learning easily misidentifies local lighting variations across distant crops as style shifts. PETAL introduces Photographic Style Augmentation (PSA), combining differentiable photo-editing operators (exposure, contrast, saturation, highlights/shadows, and tint) parameterized by \(\xi_i \in [-1, 1]\).

For an anchor image crop, small spatial jitters produce overlapping views under identical PSA settings, serving as positive pairs to build spatial invariance via intra-sample loss (\(\mathcal{L}_-\)). Simultaneously, applying inverted parameters \(f(I, \{-\xi_i\})\) to the exact same crop synthesizes hard negative counterparts with identical scene content and geometry but opposite photographic adjustments (inter-sample loss \(\mathcal{L}_+\)). To prevent intermediate features from losing absolute color statistics, an auxiliary CNN reconstructs the original luminance and chromaticity histograms from patch tokens under Hellinger distance supervision (\(\mathcal{L}_{\text{rec}}\)).

3. Adaptive 5D Neural LUT Modulation: Artifact-Free and Texture-Preserving Transfer

Traditional 3D LUTs cannot account for spatially varying effects like vignetting, while dense convolutional networks tend to induce color bleeding and texture blurring. PETAL adopts an adaptive 5D neural LUT operating over pixel colors (Lab channels) and normalized spatial physical coordinates (xy coordinates). The neural LUT consists of an encoder \(\mathcal{E}\) and decoder \(\mathcal{D}\) constructed purely with \(1 \times 1\) convolutions, ensuring strictly localized pixel operations without spatial context mixing.

In the intermediate latent space, two lightweight MLPs (\(g_\mu, g_\sigma\)) map the photographic embedding difference between style reference \(M(I_s)\) and content image \(M(I_c)\) into affine scale and bias parameters:

\[ \hat{I} = \mathcal{D}\left( \tilde{\mu} + \frac{x_c - \mu(x_c)}{\sigma(x_c) + \epsilon} \odot \tilde{\sigma} \right) \]

where \(\tilde{\mu} = \mu(x_c) + g_\mu(M(I_s)) - g_\mu(M(I_c))\) and \(\tilde{\sigma} = \sigma(x_c) + g_\sigma(M(I_s)) - g_\sigma(M(I_c))\). When the reference and content share identical photographic styles, the delta naturally vanishes, reducing the operation to an approximate identity mapping. This property guarantees stability during sequential editing and avoids hue shift in neutral regions.

Loss & Training

PETAL utilizes a stable two-stage training scheme. In Stage 1, the photographic style embedding network \(\mathcal{M}\) is optimized using:

\[ \mathcal{L}_{\mathcal{M}} = \mathcal{L}_- + 0.5 \cdot \mathcal{L}_{\text{rec}} + 0.1 \cdot \mathcal{L}_+ \]

where \(\mathcal{L}_-\) uses stop-gradient centering to align positive crops, \(\mathcal{L}_+\) separates contrary augmentation views via binary cross-entropy, and \(\mathcal{L}_{\text{rec}}\) penalizes histogram reconstruction error using Hellinger distance. In Stage 2, the embedding network \(\mathcal{M}\) is frozen. Content patches are dynamically augmented with PSA to produce paired ground-truth targets. The 5D neural LUT is trained using a weighted combination of pixel-wise MSE, LPIPS perceptual loss, embedding-based cosine style loss \(1 - \langle M(\hat{I}), M(I_s) \rangle\), and an identity mapping loss \(\| \mathcal{D}(\mathcal{E}(I_c)) - I_c \|_2^2\).

Key Experimental Results

Main Results

The authors conduct evaluations on style retrieval benchmarks (PPR10K subset, FiveK-Concept, InsRank) and style transfer benchmarks (PPR10K subset, PST50).

In photographic style retrieval, PETAL significantly outperforms general vision-language embeddings (CLIP, BLIP-2, SigLIP-2, DINOv3) and specialized image style representations (CSD, ALADIN, ColorD):

Model Category Representation Method PPR10K R@1 (%) PPR10K mAP (%) FiveK R@1 (%) FiveK mAP (%) InsRank R@1 (%) InsRank \(\rho\)
General Image CLIP 0.08 31.62 0.89 9.72 34.38 15.32
General Image DINOv3 1.48 30.81 0.02 8.42 43.75 25.31
Style Specific CSD 6.77 38.44 8.35 17.85 37.50 40.94
Style Specific ColorD 65.80 64.15 45.45 41.25 46.88 13.44
Fine-tuned DINOv3* 59.30 61.78 34.72 36.61 34.38 15.31
Ours PETAL (Ours) 74.27 71.99 52.94 51.84 62.50 52.19

In photographic style transfer quantitative benchmarks, PETAL achieves top performance in both pixel-level accuracy and structural fidelity, while running in only 0.23 seconds for a \(1080 \times 1920\) resolution image:

Method Inference Time (s) PPR10K LPIPS↓ PPR10K PSNR↑ PPR10K SSIM↑ PST50 LPIPS↓ PST50 PSNR↑ PST50 SSIM↑
SALUT 0.3117 0.2464 16.47 0.6307 0.1557 21.93 0.8733
Modflow 0.3680 0.2160 19.64 0.7914 0.1707 20.01 0.8599
Deep Preset 0.0329 0.1717 20.88 0.8137 0.1365 23.40 0.8895
Neural Preset N/A 0.1377 19.12 0.8048 0.1210 22.61 0.8975
Pixelmator Pro N/A 0.1207 19.43 0.8825 0.1526 21.60 0.8684
PETAL (Ours) 0.2306 0.1027 21.80 0.8710 0.1021 24.63 0.9272

Ablation Study

Table 3 in the paper dissects the contribution of each architectural component and loss formulation:

Ablation Configuration PPR10K R@1 (%) PPR10K mAP (%) FiveK R@1 (%) PST50 LPIPS↓ PST50 PSNR↑ PST50 SSIM↑
RGB Color Space instead of Lab 70.34 69.16 51.21 0.1068 22.22 0.9120
W/o Cross Attention 68.93 66.44 50.29 0.1120 22.24 0.9005
[CLS] Token Only 73.74 70.01 49.17 0.1131 23.11 0.9086
[CLS] + Mean Pooling 68.91 66.53 50.48 0.1149 22.49 0.9001
W/o Histogram Rec Loss \(\mathcal{L}_{\text{rec}}\) 66.98 66.06 45.86 0.1354 21.69 0.8967
W/o Intra-sample Loss \(\mathcal{L}_-\) 36.09 45.42 18.64 0.1249 23.47 0.9063
W/o Inter-sample Loss \(\mathcal{L}_+\) 67.53 65.03 51.79 0.1071 22.41 0.9045
Complete Model 74.27 71.99 52.94 0.1021 24.63 0.9272

Key Findings

  • Intra-sample loss is critical for representation stability: Removing the positive centering objective \(\mathcal{L}_-\) causes PPR10K R@1 to crash from 74.27% to 36.09%, highlighting that spatial jitter invariance is foundational to consistent style extraction.
  • Contrary augmentation blocks semantic shortcuts: Excluding the inter-sample hard negative loss \(\mathcal{L}_+\) leads to a drop in retrieval accuracy across both PPR10K and FiveK, verifying that contrasting opposite adjustments forces the network to ignore scene semantics.
  • Histograms compensate for ViT architectural bias: Dropping cross-attention guidance degrades histogram correlation (H-Corr) on PST50 from 0.5220 to 0.4436, demonstrating that attention tokens alone struggle to represent global continuous color distributions.

Highlights & Insights

  • Formulating photographic retouching as a continuous embedding space: By stepping away from coarse discrete text labels and implementing self-supervised continuous perturbations, PETAL successfully builds a dedicated representation space for photography.
  • Delta-conditioned 5D neural LUT: Injecting embedding deltas (\(M(I_s) - M(I_c)\)) via AdaIN gives the model guaranteed mathematical identity preservation when transferring identical styles, while \(1 \times 1\) convolutions ensure extreme computational efficiency and eliminate spatial blurring artifacts.
  • Smooth latent interpolation and sequential editing stability: Because the learned space is continuous, linearly interpolating between two style embeddings results in smooth, natural visual transitions, and multi-round iterative editing remains robust without degradation.

Limitations & Future Work

  • Lack of localized facial skin tone awareness: The current method models full-image and smooth spatially varying tone mappings; it does not perform localized semantic skin retouching or selective eye/lip enhancements required in studio portrait workflows.
  • Inability to synthesize non-color physical textures: Photographic styles sometimes feature analog film grain, optical halation, or chromatic aberration. The 5D LUT structure modifies per-pixel color coordinates but cannot synthesize high-frequency procedural grain.
  • Future directions: Integrating lightweight facial semantic segmentation priors for multi-region adjustment, and incorporating parametric film grain generators for comprehensive analog film simulation.
  • vs Neural Preset / Deep Preset: Prior preset methods rely on discrete preset parameter fitting or simple color discriminators, failing on high-contrast monochrome or complex cross-color grades; PETAL's self-supervised embedding provides substantially richer style sensitivity.
  • vs SA-LUT: SA-LUT adopts a 4D spatial adaptive LUT but utilizes generic CNN feature extractors, causing contrast collapse on stylized inputs; PETAL couples a 5D coordinate-aware formulation with Lab histogram cross-attention to prevent lighting distortion.
  • vs Diffusion Retouching (e.g., DiffRetouch): Diffusion approaches take several seconds and frequently introduce subtle hallucinated textural artifacts; PETAL runs in 0.23 seconds via \(1 \times 1\) neural LUTs and guarantees strict pixel-level texture preservation.

Rating

  • Novelty: ⭐⭐⭐⭐⭐ Pioneering self-supervised contrary augmentation contrastive learning coupled with an adaptive 5D neural LUT for photographic style.
  • Experimental Thoroughness: ⭐⭐⭐⭐⭐ Extensive evaluation across three retrieval benchmarks, two transfer benchmarks, user studies, ablation models, and t-SNE clustering.
  • Writing Quality: ⭐⭐⭐⭐⭐ Clear problem formulation, well-structured methodology, and consistent mathematical notation.
  • Value: ⭐⭐⭐⭐⭐ Highly practical and lightweight pipeline with strong deployment potential for real-time mobile and camera image processing.