Skip to content

Entropy-Controlled Flow Matching

Conference: ECCV 2026
arXiv: 2602.22265
Area: Generative Models / Optimal Transport Theory
Keywords: Flow Matching, Entropy Control, Mode Collapse, Schrödinger Bridge, Optimal Transport

TL;DR

Entropy-Controlled Flow Matching (ECFM) introduces an entropy rate constraint \(\frac{d}{dt}\mathcal{H}(\mu_t) \geq -\lambda\) to the standard flow matching objective. It theoretically demonstrates that this constraint is equivalent to the Schrödinger Bridge, provides provable mode-coverage certificates, and \(\Gamma\)-converges to classical optimal transport as \(\lambda \to 0\). As a purely theoretical paper in the field of flow matching, it establishes the mathematical foundations of constrained variational principles and entropy-controlled transport.

Background & Motivation

Background: Modern visual generative models (diffusion models, flow matching, rectified flow) achieve remarkable success in sample quality by transporting a base distribution to the data distribution via time-indexed measures. However, the training objectives are typically regressions of the velocity field or score function, without explicit constraints on the information geometry of the transport path.

Limitations of Prior Work: The endpoint fit of standard flow matching can mask harmful intermediate transport geometry, where the path may transition through low-entropy "bottlenecks," temporarily losing semantic modes at intermediate time steps (mode depletion). This represents mode collapse within deterministic transport models. The authors intuitively demonstrate this phenomenon using toy 8-Gaussian experiments: although standard FM achieves high-quality endpoint matching, the mass of multiple modes is depleted at intermediate steps.

Key Challenge: Optimal transport (OT) provides a canonical geometric concept of interpolation between measures (Wasserstein geodesics), but is fragile and sensitive to perturbations in high dimensions. While entropy regularization (Schrödinger Bridge) yields smoother interpolations, current flow-based generator training objectives do not explicitly impose entropy control principles.

Goal: To formalize entropy rate constraints within the variational principle of flow matching, establish a comprehensive mathematical foundation (KKT conditions, Schrödinger Bridge equivalence, \(\Gamma\)-convergence, mode-coverage certificates), and present an implementable training algorithm.

Key Insight: Starting from the continuity equation \(\partial_t \rho_t + \nabla \cdot (\rho_t v_t) = 0\), the identity \(\frac{d}{dt}\mathcal{H}(\mu_t) = \mathbb{E}_{\mu_t}[\nabla \cdot v(\cdot, t)]\) directly connects the entropy change rate to the divergence of the governing velocity field. Thus, the entropy rate constraint is equivalent to restricting the average compressibility of the velocity field.

Core Idea: Imposing an entropy rate lower-bound constraint \(\frac{d}{dt}\mathcal{H}(\mu_t) \geq -\lambda\) on the continuity equation paths in the Wasserstein space yields a constrained variational problem. Its KKT system reveals that the optimal velocity field decomposes into \(v^\star = u^\star - \nabla\varphi + \eta \nabla\log\rho\) (reference field + potential gradient + entropy multiplier × score direction).

Method

Overall Architecture

ECFM is defined as a constrained optimization problem in the Wasserstein space: minimizing the \(L^2(\mu)\) distance to a reference velocity field \(u^\star\) among paths satisfying the continuity equation and endpoint constraints, subject to the entropy rate constraint \(\dot{\mathcal{H}}(\mu_t) \geq -\lambda\).

\[ \min_{(\mu,v) \in \mathfrak{A}(\mu_0,\mu_T)} \frac{1}{2}\int_0^T \int_{\mathbb{R}^d} \|v - u^\star\|^2 d\mu_t dt \quad \text{s.t.} \quad \dot{\mathcal{H}}(\mu_t) + \lambda \geq 0 \;\; \text{a.e. } t \]

Key Designs

1. KKT Optimality System and Entropy Multiplier

By constructing a Lagrangian functional and introducing a non-negative multiplier \(\eta(t) \geq 0\) and a CE multiplier \(\varphi\), the complete KKT conditions are derived: (i) primal feasibility (CE + entropy rate constraint), (ii) dual feasibility \(\eta \geq 0\), (iii) complementary slackness \(\eta(\dot{\mathcal{H}} + \lambda) = 0\) (activating the multiplier only when the constraint is active), and (iv) the stationarity condition for \(v\), which yields the optimal velocity field decomposition: \(v^\sharp = u^\star - \nabla\varphi + \eta\nabla\log\rho\).

This decomposition offers a clear physical interpretation: \(-\nabla\varphi\) is the potential adjustments generated by the CE multiplier, and \(\eta\nabla\log\rho\) is the score-direction correction weighted by the entropy multiplier. The latter drives probability mass to diffuse from high-density to low-density regions, actively increasing the entropy.

2. Schrödinger Bridge Equivalence

When the reference path is chosen as Brownian motion (where \(u^\star\) corresponds to a drift of the form \(\frac{1}{2}\nabla\log\rho\)), the constrained optimization problem of ECFM is equivalent to the Schrödinger Bridge problem. The Lagrangian of ECFM can be rewritten in the KL divergence form \(\text{KL}(\mathbb{P} \| \mathbb{P}^{\text{ref}})\), where \(\mathbb{P}^{\text{ref}}\) is the reference path measure. This establishes a direct connection between ECFM and entropy-regularized OT.

3. \(\Gamma\)-Convergence to Classical OT (\(\lambda \to 0\))

As the entropy budget \(\lambda \to 0\), the solution of ECFM \(\Gamma\)-converges to the solution of the classical optimal transport (Benamou-Brenier formulation). \(\Gamma\)-convergence ensures that not only do the objective values converge, but any cluster point of the minimizing sequence is also a solution to the limit problem, guaranteeing that ECFM recovers the mathematical properties of OT in the limit.

4. Mode Coverage and Density Lower Bound Certificates

By formalizing the definition of mode collapse (depletion of mode mass at intermediate times), it is proven that under the entropy budget: (a) the lower bound of any mode's mass at intermediate times is jointly determined by the initial mass and \(\lambda\), (b) the density function possesses a global lower bound, and (c) these certificates are Lipschitz stable under endpoint perturbations.

5. Structural Necessity of Constraints (Failure without constraint)

By constructing explicit unconstrained FM sequences whose objective values are close to optimal but exhibit singular entropy bottlenecks and mode depletion, the authors demonstrate that without entropy constraints, approximately optimal FM paths can still collapse catastrophically. This implies that the entropy constraint is a structural necessity for certificate-level non-collapse guarantees.

Loss & Training

The appendix provides an implementable projected primal-dual augmented-Lagrangian update scheme: estimating the entropy rate on a discrete-time grid, increasing the multiplier for violating bins, while passing zero dual pressure to compliant bins. An adaptive \(\lambda\) scheduling strategy is also presented.

Key Experimental Results

Toy 8-Gaussian Mechanistic Verification

Method Preserved Modes Entropy-Feasible Bin Ratio Final MMD
Standard FM <8/8 Low Decent
ECFM (\(\lambda=1.0\)) 8/8 96% Better than FM
ECFM (Smaller \(\lambda\)) 8/8 Higher More Conservative

On the toy task, ECFM preserves all 8 modes and satisfies the entropy rate constraint for 96% of the time bins, while achieving a final MMD superior to unconstrained FM. A smaller \(\lambda\) leads to more conservative transport (yielding higher entropy).

⚠️ This is a purely theoretical paper. The main contributions lie in the mathematical foundations and toy verifications; no large-scale image generation experiments (e.g., ImageNet FID) are provided.

Highlights & Insights

  • Entropy rate constraint is an important complement to flow matching theory: Amidst the flourishing development of FM/rectified flow, this paper points out a neglected structural issue from the perspective of information geometry—good endpoint matching does not equate to a good path.
  • The velocity field decomposition \(v^\star = u^\star - \nabla\varphi + \eta\nabla\log\rho\) is the core result: It explicitly demonstrates how entropy control is achieved via an additional drift along the score direction, which is mathematically elegant.
  • Schrödinger Bridge equivalence provides a theoretical bridge: It connects ECFM to the rich literature of SB/entropy-regularized OT, enabling the direct transfer of existing analytical tools.
  • \(\Gamma\)-convergence guarantees compatibility: Recovering classical OT as \(\lambda \to 0\) indicates that ECFM is a generalization rather than a replacement, containing OT as a special case.
  • Certificate-level guarantees represent a distinctive contribution: Unlike heuristic anti-collapse methods (such as gradient penalties), ECFM provides verifiable quantitative lower bounds.

Limitations & Future Work

  • As a purely theoretical paper, evaluation is limited to the toy 8-Gaussian dataset, leaving a significant gap to large-scale image generation.
  • Entropy rate estimation in high dimensions is extremely challenging—since \(\mathcal{H}(\mu_t)\) itself requires density estimation, computing \(\dot{\mathcal{H}}\) reliably during training is a major engineering hurdle.
  • The bilevel optimization (inner and outer loops) of the Augmented Lagrangian increases training complexity.
  • The choice of \(\lambda\) lacks theoretical guidance—how to automatically set the entropy budget based on the dataset and model scale remains an open question.
  • vs Rectified Flow: Rectified flow straightens paths through iterative reflow, while ECFM prevents paths from over-compressing through entropy constraints—the two are complementary.
  • vs Schrödinger Bridge: SB typically fixes the reference Brownian motion and endpoints, whereas ECFM is a constrained variational generalization of SB—allowing the preservation of any FM reference drift \(u^\star\) and activating the entropy constraint only when necessary.
  • vs Gradient Penalty / Spectral Normalization: These are heuristic approaches to prevent mode collapse in GANs, whereas ECFM provides a theoretically guaranteed alternative.

Rating

  • Novelty: ⭐⭐⭐⭐⭐ (First to introduce entropy rate constraints to the FM variational principle, with a complete theoretical system)
  • Experimental Thoroughness: ⭐⭐ (Toy experiments only, lacks large-scale generative verification)
  • Writing Quality: ⭐⭐⭐⭐⭐ (Mathematically rigorous, complete proofs, clear structure)
  • Value: ⭐⭐⭐⭐ (Provides an important theoretical foundation and anti-collapse guarantees for flow matching, though still far from practical utility)