Straight-Path Flow Matching for Incomplete Multi-View Clustering¶
Conference: ECCV 2026
Paper: ECCV Official
Code: CVF/ECCV Repository
Area: Image Generation
Keywords: Incomplete Multi-View Clustering / Flow Matching / Straight Path / ODE Transport / Cross-View Imputation
TL;DR¶
Addressing cross-cluster drift and redundant trajectory overhead in diffusion-based incomplete multi-view clustering caused by isotropic Gaussian noise initialization, this paper proposes a deterministic ODE transport framework based on Straight-Path Flow Matching coupled with cluster-level contrast and entropy-weighted simplex alignment, achieving structure-preserving imputation and robust clustering under single-step integration.
Background & Motivation¶
Incomplete Multi-View Clustering (IMVC) seeks to partition data into semantic clusters when samples suffer from missing modalities or views caused by sensor faults, occlusions, or transmission drops. Early methodologies addressed missing attributes through autoencoder reconstructions, matrix factorizations, or graph-based collaborative filtering to align latent spaces. Recent paradigms have shifted toward end-to-end generative completion, where diffusion models synthesize missing views from Gaussian noise via conditional reverse stochastic differential equations (SDEs). Under joint end-to-end training objectives, generative trajectories directly dictate the geometric evolution of the underlying latent representations.
However, the stochastic nature of diffusion models introduces fundamental structural conflicts with clustering objectives. First, diffusion generation enforces an isotropic, class-agnostic Gaussian prior \(\mathcal{N}(0, I)\) as its starting point. The generative path must traverse a protracted trajectory from random noise to meaningful representations, failing to exploit the spatial proximity between aligned cross-view latent embeddings. Second, cluster separability in diffusion models requires stringent asymptotic conditions: cluster structure only emerges in low-noise regimes where noise variance satisfies \(\sigma_t < \delta / 2\). Under finite integration steps or moderate-to-high noise levels, support distributions overlap substantially across clusters, triggering severe cross-cluster drift.
This paper revisits probability path design in generative IMVC, asserting that cross-view completion is essentially structure-preserving transport between paired latent manifolds rather than unconditional synthesis. Core idea: Construct a deterministic ODE transport field between observed and missing views via Straight-Path Flow Matching, mathematically proving its self-shortening shortcut property and finite-step cluster separability, integrated with cluster-level contrastive alignment and entropy-weighted simplex regularization for end-to-end clustering.
Method¶
Overall Architecture¶
The proposed framework comprises three synergistic components: single-view autoencoder reconstruction networks, a straight-path flow matching imputation module, and a cross-view alignment module (incorporating cluster-level contrastive alignment and entropy-weighted simplex alignment). Dual-view inputs with missing masks are first projected into compact latent spaces via view-specific encoders. When a view is missing, the flow matching module integrates a deterministic Ordinary Differential Equation (ODE) along Euclidean linear shortcuts to transport the observed latent embedding into the missing space. Real and imputed representations are subsequently fed into clustering and fusion heads to compute joint contrastive and centered log-ratio (CLR) entropy losses, delivering robust bilingual cross-view completion and consistent clustering under single-step integration (\(\text{Step} = 1\)).
%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
A["Dual-View Input X1, X2<br/>with Missing Masks"] --> B["View-Specific Autoencoders<br/>Extract Latent Embeddings Z1, Z2"]
B --> C["Straight-Path Flow Matching<br/>Deterministic ODE Transport"]
C --> D["Bidirectional Consistency Constraint<br/>Finite Endpoint Calibration"]
D --> E["Feature Fusion & Clustering Heads<br/>Predict Probabilities Y1, Y2, Yh"]
E --> F["Cross-View Dual Alignment<br/>Cluster Contrast & Simplex Entropy"]
F --> G["Clustering Output & Representations"]
Key Designs¶
1. Straight-Path Flow Matching Imputation: Deterministic ODE Over Stochastic Noise Degradation To eliminate the trajectory redundancy and manifold disruption inherent in Gaussian diffusion, the authors introduce a deterministic ODE transport mechanism via flow matching. Given paired latent embeddings \((z_1, z_2)\), a straight interpolation path is defined as \(x_t = (1-t)z_1 + t z_2\) for \(t \in [0, 1]\). The target vector field corresponds to the shortest Euclidean displacement: $\(v_{\text{target}}(x_t, t) = \frac{d x_t}{dt} = z_2 - z_1\)$ A neural vector field \(\mathcal{F}(x_t, t)\) is trained via mean squared error regression: $\(\mathcal{L}_{\text{flow-pred}} = \mathbb{E}_{t \sim \mathcal{U}(0,1), (z_1, z_2)}\left[\|\mathcal{F}(x_t, t) - (z_2 - z_1)\|^2\right]\)$ This formulation yields two theoretical advantages: first, an adaptive shortcut trajectory whose length \(\|z_2 - z_1\|\) dynamically shrinks toward zero as cross-view representations align; second, strict cluster separability across arbitrary finite steps, because deterministic trajectories remain confined within data manifolds without intersecting, preventing noise-induced cluster overlap.
2. Bidirectional Completion Consistency: Invertible Endpoint Regression Although the velocity field theoretically prescribes linear paths, finite numerical integration steps can introduce discretization drift. Exploiting the time-reversibility of ODE flow matching, reverse integration (\(s = 1 - t\) from \(z_2\) back to \(\hat{z}_1\)) exhibits opposite velocity \(-\mathcal{F}\). Forward and backward trajectories satisfy: $\(\hat{z}_2 = z_1 + \int_0^1 \mathcal{F}(x_t, t)\,dt, \quad \hat{z}_1 = z_2 + \int_0^1 -\mathcal{F}(x_{1-t}, 1-t)\,ds\)$ A bidirectional completion consistency loss enforces terminal alignment at both boundaries: $\(\mathcal{L}_{\text{flow-comp}} = \mathbb{E}\left[\|\hat{z}_2 - z_2\|^2 + \|\hat{z}_1 - z_1\|^2\right]\)$ This constraint enables bidirectional completion using a single velocity network (unlike diffusion baselines requiring two separate models) and pins trajectory endpoints tightly, allowing single-step Euler integration (\(\text{Step} = 1\)) to achieve high imputation fidelity.
3. Cluster-Level Contrast and Entropy-Weighted Simplex Alignment: Coordinated Geometry Calibration To sharpen cluster margins during feature imputation, dual alignment objectives are imposed in the prediction space. First, a cluster-level InfoNCE contrastive loss \(\mathcal{L}_{\text{cluster}}\) maximizes cosine similarity between categorical distributions \(y_1, y_2 \in \mathbb{R}^K\): $\(\mathcal{L}_{\text{cluster}} = -\frac{1}{N}\sum_{i=1}^N \log \frac{\exp(\cos(y_1^{(i)}, y_2^{(i)})/\tau)}{\sum_{j=1}^N \exp(\cos(y_1^{(i)}, y_2^{(j)})/\tau)}\)$ Second, addressing simplex geometry distortion caused by standard Euclidean probability averaging, the Centered Log-Ratio (CLR) transformation projects distributions into real space, weighting each branch inversely by its prediction entropy to form target distribution \(y^*\). A Kullback-Leibler divergence penalizes the fused prediction \(y_h\): $\(\mathcal{L}_{\text{entropy}} = \mathrm{KL}(y_h \parallel y^*)\)$ This mechanism suppresses uncertain predictions and stabilizes the probability simplex, providing well-separated representations that reinforce flow matching stability.
Loss & Training¶
The end-to-end composite training objective balances reconstruction, flow matching, and alignment: $\(\mathcal{L} = \mathcal{L}_{\text{rec}} + \lambda_1 \mathcal{L}_{\text{flow-pred}} + \lambda_2 \mathcal{L}_{\text{flow-comp}} + \lambda_3 \mathcal{L}_{\text{cluster}} + \lambda_4 \mathcal{L}_{\text{entropy}}\)$ where \(\mathcal{L}_{\text{rec}}\) penalizes autoencoder reconstruction errors across observed views. Training utilizes the Adam optimizer with initial learning rate \(1 \times 10^{-3}\) for 200 epochs and batch sizes of 128 or 256. Notably, single-step integration (\(\text{Step} = 1\)) is employed throughout training and evaluation, keeping computational overhead minimal.
Key Experimental Results¶
Main Results¶
Quantitative evaluations across five public benchmarks under missing rates \(\tau \in \{0.1, 0.3, 0.5\}\) comparing against diffusion SOTA (DCG), contrastive baselines (ProImp, DCP), and generative models:
| Dataset | Missing Rate \(\tau\) | Metric | Ours | Runner-up / Prev. SOTA (DCG [53]) | Gain |
|---|---|---|---|---|---|
| Synthetic3D | 0.1 | ACC / NMI / ARI (%) | 94.17 / 78.66 / 83.51 | 91.23 / 71.35 / 76.05 | +2.94 / +7.31 / +7.46 |
| Synthetic3D | 0.3 | ACC / NMI / ARI (%) | 91.17 / 69.56 / 75.56 | 88.00 / 64.11 / 68.48 | +3.17 / +5.45 / +7.08 |
| Synthetic3D | 0.5 | ACC / NMI / ARI (%) | 88.17 / 63.78 / 68.26 | 87.67 / 63.23 / 67.52 | +0.50 / +0.55 / +0.74 |
| CUB | 0.1 | ACC / NMI / ARI (%) | 86.00 / 78.19 / 72.28 | 82.23 / 77.70 / 69.21 | +3.77 / +0.49 / +3.07 |
| CUB | 0.3 | ACC / NMI / ARI (%) | 82.67 / 72.80 / 66.31 | 77.17 / 71.35 / 59.85 | +5.50 / +1.45 / +6.46 |
| CUB | 0.5 | ACC / NMI / ARI (%) | 80.50 / 71.74 / 63.15 | 75.50 / 72.21 / 59.12 | +5.00 / -0.47 / +4.03 |
| HandWritten | 0.1 | ACC / NMI / ARI (%) | 85.30 / 82.78 / 76.72 | 82.75 / 82.63 / 74.88 | +2.55 / +0.15 / +1.84 |
| HandWritten | 0.3 | ACC / NMI / ARI (%) | 86.50 / 82.09 / 77.04 | 82.70 / 80.54 / 73.96 | +3.80 / +1.55 / +3.08 |
| HandWritten | 0.5 | ACC / NMI / ARI (%) | 83.40 / 76.26 / 70.76 | 80.80 / 76.21 / 70.45 | +2.60 / +0.05 / +0.31 |
| LandUse-21 | 0.1 | ACC / NMI / ARI (%) | 29.10 / 31.95 / 15.15 | 27.52 / 31.36 / 14.57 | +1.58 / +0.59 / +0.58 |
| LandUse-21 | 0.3 | ACC / NMI / ARI (%) | 28.19 / 31.71 / 14.55 | 27.33 / 32.09 / 14.47 | +0.86 / -0.38 / +0.08 |
| Fashion | 0.1 | ACC / NMI / ARI (%) | 96.07 / 91.99 / 91.78 | 95.83 / 91.29 / 91.19 | +0.24 / +0.70 / +0.59 |
| Fashion | 0.3 | ACC / NMI / ARI (%) | 93.66 / 87.93 / 87.07 | 93.13 / 86.99 / 86.00 | +0.53 / +0.94 / +1.07 |
| Fashion | 0.5 | ACC / NMI / ARI (%) | 90.09 / 82.83 / 80.58 | 90.04 / 82.25 / 79.99 | +0.05 / +0.58 / +0.59 |
Ablation Study¶
Component ablation on the HandWritten dataset (\(\tau = 0.3\)), retaining base reconstruction loss \(\mathcal{L}_{\text{rec}}\) across all variants:
| Variant | \(\mathcal{L}_{\text{flow-pred}}\) | \(\mathcal{L}_{\text{flow-comp}}\) | \(\mathcal{L}_{\text{cluster}}\) | \(\mathcal{L}_{\text{entropy}}\) | ACC (%) | NMI (%) | ARI (%) | Impact Analysis |
|---|---|---|---|---|---|---|---|---|
| (w/o) \(\mathcal{L}_{\text{flow-pred}}\) | ✗ | ✓ | ✓ | ✓ | 73.95 | 70.92 | 61.42 | Removing velocity supervision drops ACC by 12.55% |
| (w/o) \(\mathcal{L}_{\text{flow-comp}}\) | ✓ | ✗ | ✓ | ✓ | 76.15 | 71.86 | 63.38 | Omitting endpoint consistency degrades ACC by 10.35% |
| (w/o) Both Flow Terms | ✗ | ✗ | ✓ | ✓ | 77.35 | 71.89 | 64.16 | Degrades to pure alignment without generative transport |
| (w/o) \(\mathcal{L}_{\text{cluster}}\) | ✓ | ✓ | ✗ | ✓ | 27.75 | 37.13 | 20.05 | Lacking cluster guidance collapses latent geometry (\(-58.75\%\)) |
| (w/o) \(\mathcal{L}_{\text{entropy}}\) | ✓ | ✓ | ✓ | ✗ | 77.20 | 71.79 | 66.69 | Disabling entropy weighting reduces ACC by 9.30% |
| (w/o) Both Alignments | ✓ | ✓ | ✗ | ✗ | 28.70 | 17.67 | 7.75 | Severe manifold collapse without explicit clustering losses |
| Full Model | ✓ | ✓ | ✓ | ✓ | 86.50 | 82.09 | 77.04 | Optimal configuration achieving superior metrics |
Key Findings¶
- Clustering Alignment as Representational Foundation: Ablation indicates \(\mathcal{L}_{\text{cluster}}\) is essential (removing it reduces ACC from 86.50% to 27.75%), proving that class-separable manifolds are required before flow matching can effectively transport intra-cluster features.
- Flow Matching Imputation Outperforms Pure Alignment: Removing flow terms drops ACC to 77.35%, whereas restoring linear ODE transport elevates accuracy to 86.50% (+9.15%), confirming the efficacy of explicit continuous transport over naive feature pulling.
- Step Invariance Confirms Theoretical Separability: Increasing flow integration steps from 1 to 5, 10, or 25 yields negligible metric variance, corroborating Corollary 2 that straight paths ensure cluster separability even under single-step Euler integration.
Highlights & Insights¶
- Deterministic Transport Eliminates Random Diffusion Drift: Replaces isotropic Gaussian priors with straight linear ODE paths between observed representations, establishing self-shortening shortcut dynamics.
- Single Velocity Network Enables Bidirectional Inversion: Leverages ODE reversibility to perform bidirectional view imputation with one velocity field, reducing parameter overhead by 50% relative to dual-diffusion setups.
- CLR Simplex Calibration Prevents Probability Distortion: Employs centered log-ratio transformations and entropy weighting to aggregate multi-view distributions while preserving geometric simplex topology.
Limitations & Future Work¶
- Reliance on Paired Samples for Velocity Supervision: Computing target velocity \(z_2 - z_1\) requires fully observed pairs during training; performance under severe missing regimes (\(\tau \ge 0.7\)) with scarce complete pairs requires further study.
- Extension to Arbitrary Multi-View Numbers: The current formulation focuses primarily on two-view setups; scaling to arbitrary view counts (\(V > 2\)) via star or fully-connected transport graphs represents an open direction.
Related Work & Insights¶
- vs. Diffusion Generative Clustering (DCG [53]): DCG relies on multi-step stochastic denoising from Gaussian noise, incurring latency and cross-cluster drift under finite steps; the proposed method achieves single-step deterministic transport with strict separability.
- vs. Contrastive Imputation (Completer [21], ProImp [17]): Contrastive methods align representations without continuous probability paths; Straight-Path Flow Matching provides rigorous manifold geometry guarantees and exact generative transport.
Rating¶
- Novelty: ⭐⭐⭐⭐☆ Principled identification of diffusion clustering limitations and elegant formulation of straight-path flow matching for IMVC.
- Experimental Thoroughness: ⭐⭐⭐⭐⭐ Comprehensive benchmarks across 5 datasets, 3 missing rates, rigorous ablations, and step-invariance evaluations.
- Writing Quality: ⭐⭐⭐⭐⭐ Clear mathematical derivations, well-structured exposition, and rigorous theoretical grounding.
- Value: ⭐⭐⭐⭐☆ Delivers an efficient, single-step deterministic alternative to diffusion-based multi-modal imputation.