title: >- [Paper Note] MaterialFlow: Attribute-Disentangled Material Transfer via Trajectory-Aware Velocity Modulation description: >- [ECCV 2026][Signal Comm][Material Transfer] Inversion-free image material transfer via rectified flow velocity modulation, trajectory normalization, and attribute-disentangled control. tags: - ECCV 2026 - Image Editing - Material Transfer - Rectified Flow - Diffusion Models date: 2026-09-19 content_hash: 2d92e769b5c30f3a
MaterialFlow: Attribute-Disentangled Material Transfer via Trajectory-Aware Velocity Modulation¶
Conference: ECCV 2026
Paper: ECCV Official
Code: https://github.com/Sung-Lin/MaterialFlow
Area: Image Generation
Keywords: Material Transfer, Rectified Flow, Inversion-Free Editing, Trajectory Normalization, Attribute Disentanglement
TL;DR¶
To eliminate structural distortion and trajectory drift caused by feature entanglement and inversion accumulation in material transfer, MaterialFlow introduces an inversion-free trajectory-aware velocity modulation framework alongside a tri-attribute disentanglement paradigm for color, texture, and structural patterns on pre-trained Rectified Flow models.
Background & Motivation¶
Material transfer aims to re-render the surface appearance of an object using a reference exemplar while strictly preserving the underlying geometry and semantic identity of the source object. Conceptually serving as a form of digital alchemy, this capability is central to digital content creation, augmented reality previews, and 3D asset generation workflows. However, in physical scenes, material properties are inherently coupled with local illumination and macro geometry. Existing 2D transfer methods typically fall into two regimes: optimization-based fine-tuning methods that incur prohibitive per-task computation and suffer from overfitting, or training-free feature-injection frameworks like ZeST and MaterialFusion that rely heavily on auxiliary adapters or depth maps yet still suffer from texture leakage across boundaries and structural distortion.
The emergence of flow-based generative architectures, notably Rectified Flow (RF), models generative trajectories via deterministic ordinary differential equations (ODEs), providing a straight-line latent path well suited for precise velocity field modulation. Nevertheless, directly deploying RF to exemplar-based material transfer encounters two fundamental bottlenecks. First, existing flow editing methods rely primarily on inversion; because deterministic ODE transport compounds local numerical reconstruction errors step by step, inverting both the source object and reference exemplar simultaneously amplifies trajectory drift and obliterates fine object details. Second, existing pipelines encode reference materials as a single entangled representation, causing heterogeneous factors—such as chromatic hue, microscopic high-frequency roughness, and meso-scale structural patterns—to compete aggressively during generative dynamics, leading to an all-or-nothing failure mode where either transfer fails or geometry is destroyed.
This paper tackles these challenges by circumventing explicit inversion, constructing forward transport differences on interpolated latent trajectories, and dynamically recalibrating the velocity field via displacement norms while decoupling material factors across the frequency and spatial domains. Core idea: MaterialFlow provides a training-free framework that drives generative dynamics through inversion-free velocity differences, balances exemplar guidance against source semantics via trajectory-aware velocity normalization, and decouples reference materials into color, texture, and structural patterns for fine-grained modulation.
Method¶
Overall Architecture¶
MaterialFlow operates natively within the latent space of pre-trained flow backbones (e.g., FLUX.1-dev) in an entirely training-free and inversion-free manner. Given a source object image \(I_O\) and a material exemplar \(I_M\), the pipeline first applies Attribute-aware Disentanglement to decouple \(I_M\) into a color exemplar \(I_C\), a texture exemplar \(I_T\), and a structural pattern exemplar \(I_P\), which are then encoded into initial latents \(\hat{Z}^C_0, \hat{Z}^T_0, \hat{Z}^P_0\). Over an effective reverse-time diffusion editing interval \([n_{\max}, n_{\min}]\), forward linear interpolation paths are constructed from the source latent. At each step, target condition velocity differences and trajectory-normalized attribute velocity differences are computed and aggregated, advancing the latent state via first-order ODE integration to yield the re-rendered object.
%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
A["Input: Source Object Image + Reference Material Exemplar"] --> B["Attribute-aware Disentanglement<br/>Frequency filtering and patch extraction decouple C / T / P"]
B --> C["Inversion-Free Velocity Construction<br/>Linear trajectory interpolation yields task velocity differences"]
C --> D["Trajectory-Aware Velocity Normalization<br/>Dynamically balances exemplar displacement norms with source trajectory"]
D --> E["Attribute Velocity Aggregation<br/>Synthesizes overall velocity difference Delta V"]
E --> F["First-Order ODE Integration<br/>Updates latent state over editing interval to generate final image"]
Key Designs¶
1. Inversion-Free Velocity Construction: Preventing Accumulated Reconstruction Drift Conventional inversion pipelines map images back to Gaussian noise by solving reverse ODEs; however, discrete integration errors accumulate monotonically along deterministic paths, corrupting the latent trajectory. In exemplar material transfer, inverting both the source object and material exemplar introduces dual-source reconstruction errors that compound catastrophically (as evidenced by the severe drift of dual inversion over single inversion). MaterialFlow sidesteps inversion by directly constructing latent trajectories via Rectified Flow's native linear interpolation: $\(Z^{\text{src}}_t = (1-t)Z^{\text{src}}_0 + t N_t, \quad N_t \sim \mathcal{N}(0, I)\)$ At timestep \(t\), the difference between velocity predictions at the target state \(Z^{\text{tar}}_t = Z^{\text{MF}}_t + Z^{\text{src}}_t - Z^{\text{src}}_0\) and the source state defines the base task guidance: $\(\Delta V^{\text{tar}}_t = V(Z^{\text{tar}}_t, t) - V(Z^{\text{src}}_t, t)\)$ This formulation halves the required numerical function evaluations (NFEs) from two evaluations to one and fundamentally cuts off error accumulation, preserving original object geometry.
2. Trajectory-Aware Velocity Normalization: Dynamic Calibration Against Flow Imbalance When introducing visual guidance from an external material exemplar, the feature distribution of the reference often differs drastically from the source object. Uncalibrated exemplar velocity differences tend to dominate or destabilize the source trajectory, leading to either structural collapse when exemplar velocity norms overwhelm the source, or negligible transfer when exemplar velocity norms vanish. MaterialFlow resolves this imbalance by dynamically scaling the exemplar velocity according to the ratio of source-to-exemplar latent displacement norms: $\(\lambda^a_t = \frac{\|Z^{\text{src}}_t - Z^{\text{src}}_0\|}{\|\hat{Z}^a_t - \hat{Z}^a_0\| + \epsilon}\)$ $\(\Delta V^a_t = \lambda^a_t V(\hat{Z}^a_t, t) - V(Z^{\text{src}}_t, t)\)$ where \(\epsilon > 0\) ensures numerical stability. This dynamic normalization synchronizes exemplar velocity updates with the intrinsic evolution pace of the source object, preventing exaggerated non-linear distortions (e.g., dragon-like warping artifacts) and maintaining structural fidelity.
3. Attribute-Aware Disentanglement: Decoupling Color, Texture, and Structural Patterns Holistic material exemplars superimpose low-frequency color, high-frequency surface roughness, and meso-scale repetitive motifs within a single entangled representation. MaterialFlow establishes dedicated physical and frequency-domain decoupling mechanisms: - Color Disentanglement: Extracts low-frequency components via 2D Fourier low-pass filtering \(I_M^{\text{low}} = \mathcal{F}^{-1}(H_{\text{low}} \odot \mathcal{F}(I_M))\), identifies a representative chromatic vector \(c_{\text{rep}}\) in CIELab space, and aligns it with the source object's low-frequency luminance \(Y_{\text{obj}}^{\text{low}}\) via histogram matching to form \(I_C\), transferring color while preserving original shading; - Texture Disentanglement: Employs a complementary high-pass filter \(H_{\text{high}} = 1 - H_{\text{low}}\) to suppress chromatic information, isolating high-frequency surface roughness, glossiness, and granularity into \(I_T = I_M^{\text{high}}\); - Structural Pattern Disentanglement: Captures meso-scale repetitive surface arrangements (e.g., wood grain, spots, or marbling) by extracting spatially coherent patches via operator \(\mathcal{P}(I_M)\) to form \(I_P\), eliminating global shape biases. The disentangled exemplars yield latents \(\hat{Z}^C_0, \hat{Z}^T_0, \hat{Z}^P_0\), which are individually normalized and combined into \(\Delta V^{\text{ref}}_t = \sum_{a \in \{C, T, P\}} \Delta V^a_t\).
Loss & Training¶
MaterialFlow is an entirely training-free inference-time framework with no trainable parameters. Powered by pre-trained FLUX.1-dev, the full sampling trajectory uses 28 timesteps, with the active velocity modulation interval restricted to a window of 10 effective steps (\(n_{\max} - n_{\min}\)). The latent state updates via first-order Euler integration: $\(Z^{\text{MF}}_{t_{i-1}} \leftarrow Z^{\text{MF}}_{t_i} + (t_{i-1} - t_i) (\Delta V^{\text{tar}}_{t_i} + \Delta V^{\text{ref}}_{t_i})\)$ Restricting editing to this calibrated temporal interval maintains inference efficiency while ensuring natural boundary transitions.
Key Experimental Results¶
Main Results¶
Quantitative evaluations are conducted on the Material Transfer Benchmark (MTB), consisting of 30 object images and 30 reference material images evaluated under a strict training-free protocol. Metrics include: (1) VQA-based transfer quality evaluated with Qwen3-VL-30B-A3B across Color, Texture, Pattern, and their average (Avg.); (2) a user study reporting Object preservation, Material alignment, and Overall preference.
| Method | Auxiliary Modules (Aux.) | Color Alignment (Color↑) | Texture Alignment (Texture↑) | Pattern Alignment (Pattern↑) | Average Quality (Avg.↑) | Object Preservation (Object↑) | Material Match (Material↑) | Overall Preference (Overall↑) |
|---|---|---|---|---|---|---|---|---|
| IP-Adapter + SDXL | ✓ | 41.11% | 39.00% | 32.22% | 37.44% | 9.52% | 6.90% | 6.19% |
| ProSpect | – | 21.44% | 12.33% | 5.78% | 13.18% | 22.38% | 0.71% | 1.67% |
| ZeST | ✓ | 58.11% | 57.00% | 49.56% | 54.89% | 54.76% | 22.62% | 48.57% |
| MaterialFusion | ✓ | 41.33% | 38.00% | 31.78% | 37.04% | 33.33% | 13.10% | 28.81% |
| MaTe | – | 67.00% | 40.00% | 20.44% | 42.48% | 68.81% | 2.14% | 10.00% |
| MaterialFlow (Ours) | – | 79.89% | 42.67% | 34.78% | 52.45% | 83.81% | 20.48% | 59.29% |
Ablation Study¶
Ablations confirm the necessity of attribute-aware disentanglement, trajectory-aware normalization, and decoupled attribute guidance scales (visualized in Figure 6 and evaluated in the text):
| Config | Performance & Structural Integrity | Note |
|---|---|---|
| MaterialFlow (full model) | High object fidelity (83.81%) with balanced, accurate multi-attribute transfer | Achieves 59.29% overall user preference |
| w/o Attribute Disentanglement | Degrades to entangled homogeneous injection; severe attribute competition | Subdued details and blurred attribute boundaries |
| w/o Trajectory Normalization | Severe trajectory imbalance causes dragon-like geometric artifacts on source | Exemplar velocity overwhelms source structural dynamics |
| Increased Color Guidance | Elevates chromatic saturation without disturbing structural shading | Validates independent orthogonal color velocity control |
| Increased Texture / Pattern Guidance | Sharpens surface roughness and meso-scale markings while preserving geometry | Demonstrates fine-grained user-steerable material editing |
Key Findings¶
- Unrivaled Object Preservation: MaterialFlow achieves an 83.81% Object score in user studies, outperforming leading baselines like ZeST (54.76%) and MaTe (68.81%), validating that inversion-free velocity modulation eliminates cross-attention geometric degradation.
- Superior Overall Preference Without Auxiliary Networks: MaterialFlow secures the highest Overall score (59.29%)—surpassing ZeST by 10.72 percentage points—without requiring ControlNet, depth predictors, or auxiliary encoders.
- High Zero-Shot Generalizability: Under strict training-free evaluation, fine-tuning baselines (ProSpect and MaTe) collapse to 1.67% and 10.00% overall preference, highlighting the fragility of optimization-based methods compared to deterministic velocity manipulation.
Highlights & Insights¶
- Inversion-Free Latent Transport: Avoiding explicit ODE backward solving eliminates error accumulation while halving NFEs, offering a stable blueprint for exemplar editing in flow architectures.
- Norm-Ratio Dynamic Rebalancing: Using the relative displacement ratio \(\lambda_t\) to modulate exemplar velocity provides an elegant, physically intuitive mechanism to prevent cross-condition dominance.
- Tri-Attribute Frequency-Spatial Decomposition: Orthogonally decomposing material into low-frequency color, high-frequency texture, and meso-scale patches replaces monolithic latent injection with granular, modular controllability.
Limitations & Future Work¶
- Author-Acknowledged Limitations: In scenarios with extreme non-rigid deformation or complex 3D view-dependent anisotropy (e.g., deep relief carvings), the absence of explicit 3D geometry or surface normal guidance may yield less pronounced tactile depth than dedicated 3D pipelines.
- Deeper Observations: Frequency cutoffs and patch extraction operators rely on fixed spatial/frequency parameters, which may require adaptive multi-scale tuning when handling heterogeneous materials spanning micro and macro scales.
- Future Directions: Integrating adaptive spectral decomposition and lightweight normal priors could extend trajectory-aware velocity modulation to multi-view 3D Gaussian Splatting (3DGS) material editing.
Related Work & Insights¶
- vs ZeST / MaterialFusion: Both inject features via diffusion cross-attention and rely on external depth/ControlNet modules, often leaking texture across edges; MaterialFlow is completely training-free and module-free, editing via native velocity fields in flow latent space.
- vs MaTe / ProSpect: Both degrade sharply when additional fine-tuning is disallowed; MaterialFlow natively supports zero-shot, robust material transfer.
- vs FlowEdit: While FlowEdit pioneers inversion-free text editing in flow models, exemplar conditioning risks catastrophic geometric collapse; MaterialFlow solves this through trajectory-aware normalization and tri-factor attribute disentanglement.
Rating¶
- Novelty: ⭐⭐⭐⭐⭐ Groundbreaking inversion-free velocity modulation and trajectory normalization framework for flow-based exemplar editing, paired with elegant tri-attribute decomposition.
- Experimental Thoroughness: ⭐⭐⭐⭐⭐ Rigorous validation via MLLM VQA automated benchmarks and human perceptual studies across diverse baselines.
- Writing Quality: ⭐⭐⭐⭐⭐ Highly coherent mathematical formulation, clear physical intuition, and exhaustive ablation analysis.
- Value: ⭐⭐⭐⭐⭐ Establishes a highly efficient, controllable, and distortion-free paradigm for generative material editing and surface re-rendering.