Raw-JPEG Adapter: Efficient Raw Image Compression with JPEG

Conference: ECCV 2026
Paper: ECCV Official
Area: Signal & Communication
Keywords: Raw Image Compression, JPEG Adapter, Invertible Pre-processing, Neural Parameter Prediction, Sensor Signal Preservation

TL;DR

To tackle the massive storage footprint of linear raw sensor images and their incompatibility with standard 8-bit JPEG pipelines, this paper introduces Raw-JPEG Adapter—a lightweight, learnable, and invertible pre-processing pipeline that predicts channel lookup tables, frequency DCT scaling, and spatial gamma maps embedded into the JPEG comment segment (<64 KB), enabling high-fidelity raw recovery with negligible decoding runtime.

Background & Motivation

Modern digital camera image sensors record incoming scene irradiance as linear raw sensor measurements with 12–14 bits of precision. Unprocessed raw sensor data preserves the full sensor dynamic range, radiometric linearity, and fine signal distributions, serving as an indispensable asset for creative post-capture editing, advanced white balance adjustment, and low-level computer vision workflows. However, widely used raw archive formats, such as Adobe Digital Negative (DNG) or 16-bit uncompressed PNG/TIFF, incur massive storage overheads—often reaching 30–50 MB per single image. This severe footprint presents an acute bottleneck for mobile photo capture, cloud gallery synchronization, and the sharing of large-scale raw computer vision datasets.

The ubiquitous lossy compression standard JPEG offers unmatched hardware-accelerated decoding support and high compression efficiency across computing ecosystems. Nevertheless, JPEG was engineered ground-up for 8-bit, gamma-corrected display images in standard sRGB space. Storing linear sensor-domain raw data directly into the JPEG pipeline results in catastrophic quality degradation: quantizing 12–14 bits down to 8 bits causes severe posterization and color banding in dark shadow regions, while JPEG's fixed discrete cosine transform (DCT) quantization tables and chroma subsampling fundamentally mismatch the physical noise characteristics and spectral statistics of raw sensors. Prior work attempted to bridge this gap through bidirectional neural ISPs or raw reconstruction from rendered sRGB images, but these schemes either suffer from irreversible non-linear tone-mapping losses, require heavy auxiliary metadata (1–2 MB), or rely on test-time optimization (e.g., implicit neural representations) requiring tens of seconds per image.

Addressing the central tension between standard codec ubiquity and sensor-fidelity preservation, this paper bypasses sRGB rendering entirely: instead of reconstructing raw data from display-ready outputs, it directly treats the raw image as the compression target and pre-conditions its distribution to conform to JPEG quantization behavior. Core idea: construct a lightweight, fully invertible pre-processing pipeline where a compact convolutional network predicts channel-wise 1D lookup tables, block-wise DCT scaling, and pixel-wise gamma maps from a raw thumbnail, serializes the parameters (<64 KB) into the standard JPEG comment segment, and reconstructs high-fidelity raw data at decode time via closed-form analytic inversion without running neural networks.

Method

Overall Architecture

The end-to-end Raw-JPEG Adapter workflow encompasses three main stages: pre-encoding adaptation, standard JPEG compression/decompression, and post-decoding inversion. During encoding, an input demosaiced RGB raw image is downsampled into a lightweight thumbnail and fed into an efficient parameter prediction network (~37K parameters). The network predicts parameters for three cascaded invertible operators: channel-wise 1D lookup tables (LuTs) for intensity re-distribution, an optional 8×8 block-wise DCT scaling matrix for frequency-domain realignment, and a spatial pixel-wise gamma map for shadow detail expansion. The adapted image is saved using a standard JPEG encoder at user-chosen quality \(Q\), while the serialized parameters are compressed via zlib and Base64 into the standard JPEG comment (COM) marker segment (<64 KB). During decoding, any standard JPEG decoder unpacks the 8-bit image, and the stored parameters are retrieved to execute closed-form mathematical inversion in reverse order, faithfully recovering the 12–14 bit linear raw signal.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Input Linear Raw Image<br/>(12-14 bit, H×W×3)"] --> B["Lightweight Predictor<br/>Thumbnail Input (~37K params)"]
    B --> C["Channel 1D LuTs<br/>Global Tone Re-balancing"]
    C --> D["Blockwise DCT Scaling (S)<br/>Aligning Sensor & Quant Tables"]
    D --> E["Pixel-wise Gamma Map (Γ)<br/>Adaptive Shadow Expansion"]
    E --> F["Standard JPEG Encoder<br/>Controlled by Quality Q"]
    B -.->|"Parameter Storage (<64 KB)"| G["JPEG COM Marker Segment"]
    F --> H["Standard JPEG Container<br/>(.jpg file format)"]
    G --> H
    H --> I["Standard JPEG Decoder<br/>Decoded 8-bit Spatial Image"]
    I --> J["Inverse Gamma Mapping<br/>Restoring Linear Dynamic Range"]
    J --> K["Inverse DCT Scaling<br/>Restoring Frequency Distribution"]
    K --> L["Inverse 1D LuT Interpolation<br/>Restoring Sensor Tone Curves"]
    L --> M["Reconstructed Raw Output<br/>(Ready for Downstream ISP/Edit)"]

Key Designs

1. Monotonic Channel-wise 1D Look-Up Tables: Rebalancing Linear Intensity Distribution

Directly truncating linear raw sensor data into 8-bit integer space concentrates the vast majority of scene radiance values within the lowest numerical bins, leading to severe quantization noise and color casts. To alleviate this, the first adaptation stage applies independent 1D lookup tables to each of the three color channels. To guarantee strict mathematical invertibility, the lookup tables are constrained to be strictly monotonic. The prediction network outputs unconstrained increments \(h_\theta\), which are passed through a Softplus activation to ensure positive derivative and cumulatively summed into an ascending sequence normalized to \([0, 1]\): $$ \mathbf{L}c(i) = \frac{\sum, \quad c \in {R, G, B} $$ Each table contains only 128 discrete sampling nodes (}^i \text{softplus}(h_{\theta, c}(j))}{\sum_{j=1}^{128} \text{softplus}(h_{\theta, c}(j))\(128 \times 3\) floats total). Forward mapping uses piecewise linear interpolation to stretch lower intensities into a more uniform distribution; inversion during decoding is achieved via fast 1D numerical interpolation, incurring virtually zero runtime.

2. Blockwise DCT Frequency Scaling: Mitigating Codec Quantization Mismatch

Standard JPEG utilizes fixed quantization matrices tuned according to the contrast sensitivity function of human perception on natural sRGB images, which heavily penalizes high frequencies and completely mismatches sensor-domain noise spectra. To counteract this domain discrepancy, the pipeline applies an \(8 \times 8\) global scaling matrix \(\mathbf{S} \in \mathbb{R}^{8 \times 8}\) to blockwise DCT representations. For each non-overlapping \(8 \times 8\) block, DCT coefficients are multiplied element-wise by \(\mathbf{S}\) prior to inverse DCT transformation back to the spatial domain: $$ \mathbf{I}^{\text{DCT}}_b = \text{IDCT}\left( \text{DCT}(\mathbf{I}^{\text{LuT}}_b) \odot \mathbf{S} \right) $$ To prevent numeric explosion or instability, the predicted parameters are bounded within \([0.5, 2.0]\) via \(\mathbf{S} = \exp(\tanh(s_\theta))\). At decode time, the scaling is inverted through element-wise division: \(\mathbf{I}^{\text{LuT}}_b = \text{IDCT}\left( \text{DCT}(\mathbf{I}^{\text{DCT}}_b) \oslash \mathbf{S} \right)\). This frequency-domain shaping suppresses artifacts by selectively redistributing energy before standard JPEG quantization takes place.

3. Pixel-wise Spatial Gamma Mapping: Eliminating Shadow Quantization Banding

Even after global tonal reshaping, dense shadow regions remain vulnerable to visible contouring and banding artifacts under coarse quantization due to extreme local contrast. To provide spatially adaptive dynamic range compensation, the adapter predicts a smooth \(100 \times 100\) spatial gamma map \(\boldsymbol{\Gamma}\). Network activations are constrained to the safe positive range \([0.14, 7.4]\) via: $$ \boldsymbol{\Gamma}{(x,y)} = \exp\left(2.0 \cdot \tanh(g)\right) $$ The predicted map is bilinearly upsampled to full image resolution and applied element-wise as a power-law transformation: \(\mathbf{I}^{\Gamma}_{(x,y)} = \left(\mathbf{I}^{\text{DCT}}_{(x,y)}\right)^{\boldsymbol{\Gamma}_{(x,y)}}\). In the reconstruction pipeline, the operation is cleanly inverted by applying the reciprocal exponent \(\left(\mathbf{I}^{\Gamma}_{(x,y)}\right)^{1 / \boldsymbol{\Gamma}_{(x,y)}}\). This spatial flexibility provides tailored non-linear amplification in dark regions, preserving delicate gradient gradations across 8-bit bottlenecks.

4. Compact Predictor & Metadata Packaging: Enabling Zero-Overhead Deployment

The parameter prediction network employs a lightweight convolutional encoder-decoder topology featuring GELU activations, skip connections, and Efficient Channel Attention (ECA) blocks, totaling only ~37K parameters. Operating solely on downsampled raw thumbnails, the encoder inference completes in ~0.1 s on edge devices. The complete set of predicted parameters (384 LuT coefficients, 64 DCT multipliers, and 10,000 gamma map points) serializes to approximately 40 KB after zlib compression and Base64 packaging, strictly adhering to the 64 KB limit of standard JPEG COM markers. Because reconstruction relies exclusively on closed-form analytic equations, the decoding side requires zero neural network execution, restoring full raw fidelity in just 0.12 s.

Loss & Training

The framework is trained end-to-end in a self-supervised fashion on uncompressed raw images without requiring paired ground truth. To permit gradient propagation through the discrete, non-differentiable JPEG quantization operation, the training loop incorporates a differentiable JPEG simulator. Optimization minimizes a composite multi-domain objective: $$ \mathcal{L} = \lambda_{\text{L1}} \mathcal{L}{\text{L1}}(\hat{\mathbf{I}}, \mathbf{I}) + \lambda}} \mathcal{L{\text{SSIM}}(\hat{\mathbf{I}}, \mathbf{I}) + \lambda) $$ where }} \mathcal{L}_{\text{FFT}}(\hat{\mathbf{I}}, \mathbf{I\(\mathcal{L}_{\text{FFT}}\) computes \(L_1\) distances over the real and imaginary components of 2D Fast Fourier Transforms of \(\hat{\mathbf{I}}\) and \(\mathbf{I}\) to enforce spectral consistency (\(\lambda_{\text{L1}} = 1.0, \lambda_{\text{SSIM}} = 0.1, \lambda_{\text{FFT}} = 0.1\)). Training incorporates random brightness scaling and random color transformations with an explicit \(+0.05\) green offset bias, mimicking the higher green spectral sensitivity of digital camera Bayer sensors.

Key Experimental Results

Main Results

Evaluated on the Samsung Galaxy S24 Ultra test set (400 images), Raw-JPEG Adapter was benchmarked against standard JPEG and leading raw pre-processing baselines across multiple quality levels. Compression ratio (CR) is computed relative to 16-bit uncompressed raw PNGs.

Method	Quality (Q)	PSNR (dB)	SSIM (%)	MS-SSIM (%)	Bitrate BPP	Compression Ratio (CR)
Direct JPEG Storage	100	46.00	98.47	99.72	4.36	6.51×
JPEG + OLM [Chung et al.]	100	43.65	97.74	99.55	4.21	6.73×
JPEG + raw↔sRGB (CCM inversion)	100	47.65	99.25	99.86	6.09	4.70×
JPEG + Fixed Gamma (2.2)	100	48.38	99.27	99.87	5.25	5.47×
Raw-JPEG Adapter (Full)	100	49.04	99.35	99.89	5.28	5.41×
Direct JPEG Storage	75	40.75	95.40	98.62	0.56	56.42×
JPEG + OLM [Chung et al.]	75	40.33	95.17	98.56	0.55	57.20×
JPEG + raw↔sRGB (CCM inversion)	75	42.63	97.37	99.39	0.85	37.15×
JPEG + Fixed Gamma (2.2)	75	42.43	97.16	99.25	0.73	43.06×
Raw-JPEG Adapter (Full)	75	43.58	97.67	99.48	1.04	29.85×

When compared against complex metadata-based reconstruction (INF, R2LCM) and bidirectional neural ISPs (Invertible ISP), Raw-JPEG Adapter delivers the best trade-off between fidelity, file size, and decoding speed:

Category	Method	PSNR (dB)	SSIM (%)	Total / Aux BPP	Decoding Latency
Bidirectional Neural ISP	CIE XYZ Net [Afifi et al.]	20.47	81.61	2.340 / 0.000	0.301 s
Bidirectional Neural ISP	Invertible ISP [Xing et al.]	43.71	98.40	2.340 / 0.000	7.932 s
Implicit Neural Function	INF [Li et al.]	41.24	96.29	3.764 / 0.776	25.930 s
Compact Latent Metadata	R2LCM (drop 0) [Wang et al.]	46.01	98.49	3.929 / 1.701	2.815 s
Ours (JPEG Q=75)	Raw-JPEG Adapter	43.58	97.67	1.036 / 1.036	0.120 s
Ours (JPEG Q=95)	Raw-JPEG Adapter	46.22	98.60	2.345 / 2.345	0.120 s
Ours (Learned LIC-TCM)	LIC-TCM + Adapter (Tuned)	46.03	98.75	1.260 / 1.260	0.120 s

Ablation Study

Systematic ablations on operator configurations and loss components evaluated on the S24 test set at JPEG quality 75:

Config	Modification Description	PSNR (dB)	SSIM (×100)
Gamma map only	\(100 \times 100\) pixel-wise gamma operator alone	42.32	96.11
1D LuT only	\(128 \times 3\) channel-wise LuT operator alone	40.93	95.56
Gamma + LuT	Spatial-domain operator pair	42.50	97.12
Gamma + LuT + DCT	Integrating \(8 \times 8\) frequency scaling matrix	42.70	97.24
Incorporate ECA	Efficient Channel Attention in predictor	42.80	97.19
Full Model (Simulator)	All operators + ECA + differentiable simulation	43.58	97.67
Lower Gamma resolution	Gamma map reduced to \(64 \times 64\)	43.56	97.64
Fewer LuT nodes	LuT nodes reduced from 128 to 64	43.49	97.66
Shared single-channel LuT	Monolithic \(128 \times 1\) LuT for all channels	43.42	97.63
Remove FFT loss	Trained with \(L_1\) and SSIM only	43.06	97.35
Remove SSIM loss	Trained with \(L_1\) and FFT only	43.34	97.59
Remove \(L_1\) loss	Trained with SSIM and FFT only	41.72	97.62

Key Findings

• Module Contribution & Color Diversity: The pixel-wise gamma map plays the single most critical role in mitigating shadow posterization and quantization banding. In tandem with channel 1D LuTs, the reconstructed output restores over 11.39 million unique RGB color triplets (compared to only ~7,195 triplets in un-adapted JPEG), completely eliminating harsh tonal thresholds.
• Sensor Specialization vs. Cross-Camera Robustness: The \(8 \times 8\) blockwise DCT scaling provides an additional 0.2–0.3 dB gain on the training camera (Samsung S24). However, cross-camera evaluations on MIT-Adobe 5K (35 DSLRs) and NUS (8 DSLRs) reveal that omitting the camera-specific DCT module yields superior generalization, outperforming the DCT variant by up to 1.5 dB due to differences in sensor noise models.
• Bitrate Efficiency: Bjontegaard Delta rate analysis shows that Raw-JPEG Adapter attains a 27.88% bitrate saving over direct JPEG compression and 13.56% over the raw↔sRGB baseline at equivalent reconstruction PSNR.
• Codec Agnosticism: When evaluated with alternative compression schemes such as JPEG 2000 and the learned transformer-CNN compressor LIC-TCM, the adapter achieves a peak PSNR of 46.03 dB, demonstrating that pre-inversion statistical conditioning generalizes across coding architectures.

Highlights & Insights

• Invertible Adaptation Over Inverse Rendering: Instead of struggling to reconstruct raw data from lossy, tone-mapped sRGB renders, this work directly uses the raw signal as the transmission payload, adapting its distribution to seamlessly pass through 8-bit quantization bottlenecks.
• Zero-Disruption Deployment via Standard Extension Slots: By encapsulating all transformation parameters within ~40 KB inside standard JPEG COM markers, the compressed images remain 100% compliant with standard photo viewers while providing lossless-grade raw recovery in specialized pipelines.
• Closed-Form Inversion Eliminates Decoder Neural Compute: The post-decoding restoration operates entirely via matrix multiplication, analytic division, and 1D interpolation, unlocking fast sub-0.15s decoding suitable for mobile and embedded devices.

Limitations & Future Work

• Residual Block Artifacts at Ultra-Low Bitrates: At aggressive compression settings (\(Q \le 25\)), coarse JPEG block quantization still introduces faint high-frequency boundary blurring that analytic inversion cannot fully undo.
• Spatial Smoothness Assumption in Gamma Maps: Upsampling a \(100 \times 100\) gamma map via bilinear interpolation assumes illumination corrections are locally smooth, which can occasionally introduce minor inaccuracies near sharp, high-contrast backlit silhouettes.
• Future Improvements: Promising directions include exploring adaptive wavelet-based edge conditioning and expanding the adapter to multi-frame computational raw formats (e.g., Apple ProRAW / Google HDR+ raw).

Related Work & Insights

• vs. Invertible ISP [Xing et al., CVPR 2021]: Invertible ISP employs bidirectional normalizing flows between sRGB and raw, requiring ~8 s per image on a high-end GPU; Raw-JPEG Adapter decodes via analytic formulas in 0.12 s (~70× faster) with superior fidelity.
• vs. R2LCM [Wang et al., CVPR 2023 / IJCV 2024]: R2LCM stores 1–2 MB of learned latent metadata alongside sRGB images; Raw-JPEG Adapter shrinks auxiliary data down to <64 KB while removing dependency on proprietary ISP curves.
• vs. Hardware CFA Compression (JPEG XS / OLM): Traditional CFA compression addresses low-latency broadcast streaming at low compression ratios; this work targets multi-gigabyte photography storage and dataset distribution, reducing file sizes by over 30×.

Rating

• Novelty: ⭐⭐⭐⭐☆ Bypasses the traditional inverse ISP paradigm by introducing a mathematically closed, learnable pre-quantization adapter.
• Experimental Thoroughness: ⭐⭐⭐⭐⭐ Rigorous validation spanning smartphone sensors, 35 DSLR models, diverse codecs, and downstream Adobe Lightroom re-rendering pipelines.
• Writing Quality: ⭐⭐⭐⭐⭐ Clear exposition, concise formulation, and transparent architectural motivations.
• Value: ⭐⭐⭐⭐⭐ Highly impactful for mobile computational photography, cloud raw storage, and low-level computer vision data distribution.