MATCH: Flow Matching for Multi-View Anomaly Detection¶

Conference: ECCV 2026
arXiv: 2606.24375
Code: https://github.com/m-kruse98/MATCH
Area: Anomaly Detection / Multi-View Vision
Keywords: Multi-view anomaly detection, Flow Matching, normalizing flows, density estimation, industrial defect detection

TL;DR¶

MATCH is the first Flow Matching-based multi-view anomaly detection method. It utilizes continuous normalizing flows in the form of ODEs to perform density estimation on pre-trained feature spaces. By omitting the divergence term, it achieves real-time inference (18.77 FPS) and obtains state-of-the-art (SOTA) detection and segmentation performance on both Real-IAD and MANTA-Tiny benchmarks.

Background & Motivation¶

Background: The dominant paradigm in industrial anomaly detection is semi-supervised learning, where only normal samples are used during training, and anomalies deviating from the normal distribution are detected during inference. Existing high-performance methods extensively use RealNVP-based Normalizing Flows for likelihood estimation, such as DifferNet, CS-Flow, FastFlow, CFlow, and Multi-Flow.

Limitations of Prior Work: RealNVP architectures have two fundamental limitations. First, their coupling block structure requires the network to be invertible and the Jacobian determinant to be easily computable, which severely restricts the expressive capacity of the network architecture and limits transformation stacking to a small number of layers. Second, in multi-view scenarios, where image data dimensionality is extremely high (concatenation of feature maps from multiple views), RealNVP easily fails when modeling high-dimensional distributions. The paper demonstrates this intuitively in a "Two Moons" toy experiment: when 2D moon data is embedded into a $d$-dimensional space, Flow Matching correctly captures the distribution across all dimensions, whereas RealNVP begins to produce degraded density estimates (with negative BPD) as soon as $d > 2$.

Key Challenge: Multi-view anomaly detection requires capturing minor deviations (anomalies) in extremely high-dimensional feature spaces, but the representation capacity of the RealNVP architecture is constrained by its invertibility requirements, making it inadequate for accurate high-dimensional distribution modeling. Meanwhile, exact likelihood estimation requires computing the divergence term, which heavily increases inference costs and hinders real-time industrial deployment.

Goal: (1) Replace RealNVP with Flow Matching for density estimation in multi-view anomaly detection; (2) omit the divergence term to increase inference speed while maintaining detection accuracy; (3) perform the first comprehensive benchmarking on the MANTA-Tiny dataset.

Key Insight: As a modern training paradigm for continuous normalizing flows (CNFs), Flow Matching does not require network invertibility and can use arbitrary generic architectures (e.g., ResNet, Transformer), making it inherently suitable for high-dimensional data modeling. Its ODE formulation supports both the generative direction (noise $\rightarrow$ data) and the density estimation direction (data $\rightarrow$ Gaussian space); the latter is perfectly suited for computing anomaly scores.

Core Idea: Map multi-view features to a standard Gaussian space using the ODE of Flow Matching, utilize the negative log-likelihood of samples in the Gaussian space (essentially the Mahalanobis distance) as the anomaly score, and omit the divergence term to accelerate inference by over 3 times without accuracy degradation.

Method¶

Overall Architecture¶

The overall pipeline of MATCH consists of "frozen feature extraction $\rightarrow$ multi-scale feature alignment $\rightarrow$ Flow Matching latent space mapping $\rightarrow$ backward ODE integration for anomaly score computation." During training, normal multi-view images are input, and three-layer feature maps are extracted via a frozen WideResNet50. Time step $t$ and view $v$ embeddings are added. A bottleneck network aligns and fuses the multi-scale features, which are then reconstructed to their original multi-scale resolutions by a three-layer decoder. The network is trained with the OT-CFM loss to learn the vector field mapping from the data distribution to a standard Gaussian. During inference, backward ODE integration is run to integrate test samples from $t=1$ to $t=0$, yielding the mapping $x_0$ in the latent Gaussian space. The anomaly score is computed as $S(x) = \|x_0\|^2/2 + (d/2)\log(2\pi)$, generating multi-scale anomaly maps for segmentation and detection.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Multi-View Normal Images<br/>256×256"] --> B["Frozen WideResNet50<br/>Extract 3-layer Feature Maps"]
    B --> C["Bottleneck: Spatial Alignment<br/>+ Concatenation + ResNet Bottleneck Blocks"]
    C --> D["Add Time Step t and View v Embeddings"]
    D --> E["3-layer Decoder (Gradual Upsampling)<br/>Restore Multi-scale Features"]
    E --> F["OT-CFM Loss Training<br/>Learn Data → Gaussian Vector Field"]
    F --> G["Backward ODE Integration<br/>Compute Latent Space x_0"]
    G --> H["S(x) = ‖x_0‖²/2<br/>Anomaly Map → Detection/Segmentation"]

Key Designs¶

1. Flow Matching Replacing RealNVP: Fundamentally Removing Architectural Constraints

RealNVP's coupling block requires each layer to be invertible with an easily computable Jacobian, which restricts the network to specific structures (e.g., affine coupling layers) and limits depth due to representation capacity saturation. Flow Matching is fundamentally different; it learns a time-dependent vector field $u_t(x)$, defining a continuous transformation from noise $p_0$ to data $q$ via the ODE $d(\phi_t)/dt = u_t(\phi_t)$. There are no invertibility or Jacobian constraints on the network architecture. It is trained using the OT-CFM loss: $\mathcal{L} = \mathbb{E}[\|u_t^\theta(x_t) - (x_1 - x_0)\|^2]$, where $x_t = (1-t)x_0 + t \cdot x_1$ is the linearly interpolated intermediate point. This loss is a simple regression objective (predicting the straight-line direction from the current point to the target data point) and does not require backpropagation through an ODE solver. After training, forward integration ($t=0 \rightarrow 1$) is used for generation, and backward integration ($t=1 \rightarrow 0$) is used for density estimation. This architectural flexibility allows MATCH to employ ResNet-style bottlenecks and decoders with customizable depth and width. Experiments show an optimal width of 768, which far exceeds typical RealNVP configurations.

2. Multi-Scale Latent Space Density Estimation and Reverse Distillation Architecture

Performing Flow Matching directly in pixel space has been proven ineffective for anomaly detection, as the differences between normal and abnormal images are too minimal at the pixel level. MATCH learns the distribution in the latent space of a pre-trained WideResNet50, extracting the first three layers of feature maps $x_1^i \sim q^i$ (with resolutions of $256 \times 64 \times 64$, $512 \times 32 \times 32$, and $1024 \times 16 \times 16$ for $i=1,2,3$). These feature maps are first spatially aligned via a bottleneck—features of $i=1$ are downsampled using two layers of stride-2 convolutions, and those of $i=2$ are downsampled using one layer, aligning all of them to $16 \times 16$ before concatenation. They then pass through three ResNet-style bottleneck blocks (with group normalization and ReLU) to further distill the representations of normal structures. The decoder is divided into three levels $D_i$ (containing 3, 4, and 6 decoder blocks respectively). Each level uses $1 \times 1$ conv + $3 \times 3$ stride-2 transposed convolutions to upsample and gradually restore original spatial scales, outputting three anomaly maps $M_i$. This "compression-reconstruction" Reverse Distillation structure forces the bottleneck to retain only the generic structures of normal samples. Thus, abnormal samples suffer larger reconstruction deviations when reconstructed by the decoder, yielding stronger responses on the anomaly maps. Information about the time step $t$ (1024-dimensional sinusoidal positional encoding + 2-layer MLP) and view $v$ (learnable embedding + MLP) is concatenated and injected into the bottleneck output and each decoder block, keeping the network informed about the current flow path stage and the view being processed.

3. Omitting the Divergence Term: Trading for Real-Time Inference with "Incomplete Likelihood"

The standard likelihood estimation formula for Flow Matching is: $$\log p_1(x_1) = \log p_0(x_0) - \int_{0}^{1} \text{div}(u_t)(x_t) dt$$ where the divergence term must be integrated along the ODE trajectory. In practice, the divergence is computed using the Hutchinson trace estimator: $\text{div}(u_t)(x) = \mathbb{E}_z [z^T \partial_x u_t(x) z]$, requiring an extra backpropagation per step. The paper finds that on Real-IAD, the absolute value of the divergence term only accounts for 0.57% of the final score magnitude ($|\text{div}|/|\log p_0(x_0)| = 0.0057$). Adding the divergence (regardless of $M=1,2,4$ sample vectors) has virtually no impact on I-AUROC (91.172 vs 91.170). Omitting the divergence skyrockets the FPS from 5.40 ($M=1$) to 18.77, and reduces GPU memory from 10.28GB to 8.88GB. Consequently, the anomaly score of MATCH is simplified to $S(x) = \|x_0\|^2/2 + (d/2)\log(2\pi)$. Ignoring the constant term, it is essentially the squared Mahalanobis distance from $x_0$ to the standard Gaussian—the further $x_0$ is from the Gaussian center, the more likely it is an anomaly. The authors hypothesize that this is because the vector field learned by the model is highly smooth, resulting in near-zero local divergence.

4. Multi-Granularity Anomaly Score Fusion: Complementarity of Local and Global

MATCH outputs three anomaly maps $M_i$ from three decoder layers, each generated by computing $S(x)$ along the feature dimension. For the segmentation task, these are directly summed to obtain $M_{\text{segment}} = \sum M_i$, and all maps are bilinearly interpolated to the original image size. The low-level map $M_1$ excels at capturing tiny, local, low-level anomalies (e.g., scratches), while high-level maps $M_2$ and $M_3$ are more sensitive to high-level structural anomalies; stacking the three maps achieves multi-granularity coverage. For image-level detection, only $M_{\text{detect}} = M_2 + M_3$ is used (discarding $M_1$). This is because prior research has identified that high-level representations are better suited for generating a single image-level anomaly score, as low-level variations are easily triggered by normal texture changes and create noise. The image-level score is set as the maximum value of $M_{\text{detect}}$. For sample-level (multi-view object-level) detection, the maximum of the image-level scores across all views of the object is taken—meaning the entire object is classified as anomalous if any single view presents an anomaly.

Loss & Training¶

The training objective is the OT-CFM loss (formula in Key Designs 1), computed at three feature scales and then summed. The optimizer is AdamW ($\text{lr}=10^{-4}$, $\beta=(0.9, 0.95)$, weight decay = 0.01), requiring no gradient clipping or scaling (unlike RealNVP). All images are resized to $256 \times 256$, with a batch size of 8, trained for 150 epochs. During inference, the Euler method is used to solve the ODE with a step size of 0.2, totaling 5 forward passes. ODE solver ablation shows that the Euler method is not only the fastest (13.13 FPS), but also yields an I-AUROC (91.17) slightly superior to Midpoint (91.08) and RK4 (91.07), showing no accuracy-speed trade-off.

Key Experimental Results¶

Main Results¶

Real-IAD (30 industrial object classes, 5 views):

Metric	MATCH	Multi-Flow	RD4AD	EfficientAD	FastFlow
I-AUROC	91.17	90.27	89.26	77.26	79.26
S-AUROC	95.63	95.85	93.46	86.91	92.30
P-AUROC	99.24	96.47	98.81	89.64	90.18
P-AUPRO	94.76	87.91	92.70	71.31	66.99
P-AP	32.68	12.42	32.65	13.18	06.16

MATCH ranks first across all metrics except S-AUROC. P-AUPRO is 2.06 points higher than the runner-up RD4AD and 6.85 points higher than the flow-based Multi-Flow, validating that Flow Matching is better suited for anomaly segmentation in high-dimensional feature spaces than RealNVP. S-AUROC is slightly lower than Multi-Flow (-0.22), but Multi-Flow itself is specifically optimized for multi-view scenarios.

MANTA-Tiny (tiny objects, 5 main categories, multiple subcategories):

Metric	MATCH	RD4AD	Multi-Flow	SimpleNet	EfficientAD
P-AUPRO	89.66	85.59	81.76	79.42	80.83
P-AUROC	95.65	94.60	93.07	93.63	92.21
I-AUROC	90.66	88.45	—	93.19	—

P-AUPRO leads the runner-up RD4AD by over 4 percentage points. Grouped by category, Mechanics (industrial parts) achieves the highest performance (P-AUPRO 96.35), while Agriculture (natural items) yields the lowest (75.52), owing to the large variance of normal samples and ambiguous anomaly boundaries in natural categories. It is slightly below SimpleNet on I-AUROC, which the authors attribute to the difficulty of converting tiny defects in MANTA-Tiny into high image-level anomaly scores.

Ablation Study¶

Divergence Term Ablation (Real-IAD):

Method	M	I-AUROC	FPS	GPU Memory (GB)
MATCH (No Div)	—	91.172	18.77	8.88
Hutchinson	1	91.170	5.40	10.28
Hutchinson	2	91.169	3.16	10.70
Hutchinson	4	91.170	1.56	11.45
RQMC	1	91.170	—	—
RQMC	4	91.170	—	—

Adding divergence (regardless of the estimation method or number of sample vectors) yielded zero improvement in detection accuracy, but decreased FPS by at least 3.5 times. The divergence term accounts for only 0.57% of the final score, justifying its omission.

ODE Solver Ablation (Real-IAD):

Solver	Step Size	I-AUROC	FPS
Euler	0.20	91.17	18.77
Euler	0.10	91.12	6.54
Midpoint	0.20	91.08	6.55
RK4	0.20	91.07	2.48
Dopri5 (Adaptive)	—	91.06	0.48

The Euler method achieves a win-win in speed and accuracy with a large step size (0.2, requiring only 5 steps). More complex solvers do not improve AUROC (all ~91.06) but significantly reduce speed.

Network Width Ablation: Gradually increasing the feature dimension of the bottleneck and decoder from 32 to 768 leads to a near-linear improvement across all metrics, with the optimal performance reaching at a width of 768. Matching the parameter count of RD4AD to MATCH yields an I-AUROC of only 86.75 for RD4AD, confirming that the performance gains originate from the Flow Matching formulation rather than model capacity.

Key Findings¶

The divergence term can be completely omitted without performance degradation, which is critical for making MATCH practical. This implies that when Flow Matching is used for anomaly detection, precise likelihood is not required; only relatively ranked anomaly scores are necessary.
Multi-scale anomaly map fusion is key to the substantial lead in segmentation performance: low-level maps capture fine texture defects, and high-level maps perceive structural anomalies, complementing each other.
Performance on Agriculture and Groceries classes in MANTA-Tiny is significantly weaker than on Mechanics and Electronics classes, as the normal samples of the former exhibit highly diverse morphologies, and anomaly decision relies more on semantic reasoning. The authors provide several failure cases: shriveled rice grain defects (only minor color differences), soybean cracks causing the entire sample to be flagged, and false positives caused by specular highlights.
Directly evaluating the MATCH model trained on MANTA-Tiny on the full MANTA test set results in no noticeable performance drop (P-AUPRO 89.23 vs 89.66), establishing MANTA-Tiny as a good proxy benchmark for MANTA.

Highlights & Insights¶

The omission of the divergence term is the most ingenious design decision in this paper. Conventionally, Flow Matching for density estimation requires full likelihood; however, the authors experimentally demonstrate that for anomaly detection, which only requires relative ranking, "incomplete likelihood" (solely the log density of the latent Gaussian) is sufficient. This is because the norm distributions of normal and anomalous samples in the latent space are already well-separable. It effectively simplifies expensive exact density estimation into an efficient distance metric from the origin, representing an elegant compromise between theory and engineering.
Combining the Reverse Distillation architecture with Flow Matching is a natural yet previously overlooked combination. The "compression-reconstruction" structure of RD is inherently suitable for anomaly detection (as the bottleneck filters out anomalous information), while Flow Matching provides it with distribution modeling capability free of architectural constraints. Their combination leads by a margin in segmentation metrics over pure RealNVP solutions, indicating that the choice of both architectural paradigm and generative model is equally critical.
The multi-view aggregation strategy is highly simple yet effective: taking the maximum at the image level and the maximum at the sample level. This dual-level max operation introduces no additional parameters or training. This implies that in multi-view anomaly detection, the assumption "an anomaly in any view makes the object anomalous" is strong enough, negating the need for complex cross-view attention or fusion mechanisms.
The embedding of timestep and view info is reusable—injected into the bottleneck and each decoder block via sinusoidal positional encoding + MLP, allowing the network to perceive the phase of the flow path and the differences in feature distributions across views.

Limitations & Future Work¶

Insufficient handling of global structural anomalies: The authors acknowledge in the failure analysis that when multiple local parts of an object appear normal individually but are anomalous when combined (e.g., a capsule having blue caps on both ends instead of one blue and one white), MATCH's local anomaly maps fail to capture this global violation. This is because the segmentation branch focuses on local textures, and while the detection branch has a global field of view, it lacks explicit modeling of "cross-local consistency." Introducing cross-local attention or graph models might address this.
Sensitivity to illumination and pose variations: Some false positives stem from specular reflections or object rotations (lighting conditions unseen in the training set). Since the model only models the normal distribution in 2D feature space, it lacks 3D geometric priors to distinguish "surface appearance changes due to perspective changes" from "true surface defects."
S-AUROC falls short of specialized multi-view methods like Multi-Flow: Although the gap is only 0.22 points, it indicates that simple max aggregation might not be the optimal sample-level score fusion method. Factoring in complementary view information (e.g., some defects are only visible from specific views and could be inferred from other views) could yield further improvements.
I-AUROC on MANTA-Tiny lags behind SimpleNet: SimpleNet utilizes supervised training by synthesizing anomalies with added noise in the feature space. This "synthetic anomaly supervision" might offer advantages over pure density estimation in detecting tiny defects. Future work could explore the synergy between Flow Matching and synthetic anomalies.
No utilization of multi-view geometric constraints: The paper suggests that future work will introduce explicit 3D constraints or multi-view correspondences (e.g., epipolar geometry) to the Flow Matching framework.
Validated only on ImageNet pre-trained WideResNet50: Although the supplementary material tests the DINOv3 backbone (yielding slightly lower performance), more massive modern backbones (such as DINOv2 ViT-L) or direct training in pixel space remain unexplored.

vs Multi-Flow: Both are multi-view flow-based approaches, but Multi-Flow relies on RealNVP (discrete normalizing flow), bounded by coupling block structural constraints. MATCH replaces it with Flow Matching to attain higher expressive capacity and significantly better segmentation performance (P-AUPRO 94.76 vs 87.91). However, Multi-Flow performs slightly better on S-AUROC, potentially due to a more sophisticated multi-view integration strategy.
vs RD4AD: RD4AD employs a Reverse Distillation architecture for anomaly detection, but its teacher-student distillation lacks explicit density estimation. MATCH reuses its encoder-decoder structure but incorporates Flow Matching at the bottleneck to learn explicit probability distributions, yielding stronger anomaly localization capabilities. Parameter-matching experiments confirm the gains stem from FM rather than a larger model.
vs DifferNet / CS-Flow / CFlow: These single-view NF methods utilize RealNVP for density estimation and fail in multi-view scenarios. MATCH demonstrates that the distribution fitting capability of Flow Matching in high-dimensional feature spaces far exceeds RealNVP (in the same feature space, the RealNVP baseline has a BPD of up to 160.5, while MATCH is only 1.83).
vs Novel-View Synthesis-based Multi-View Methods (e.g., SplatPose, PACLD): These methods build a fixed 3D normal prototype (e.g., a 3DGS representation) and compare new-view renderings. A drawback is their inability to handle cases where normal samples exhibit high morphological variation. The density estimation approach of MATCH naturally supports learning multi-modal normal distributions.

Rating¶

Novelty: ⭐⭐⭐⭐ First to introduce Flow Matching into multi-view anomaly detection; the discovery of the omittability of the divergence term has practical value. However, the core architecture (RD + FM) is a combination of two existing modules, and the novelty at the methodology level is moderately high.
Experimental Thoroughness: ⭐⭐⭐⭐⭐ Comprehensive evaluation on two large datasets + 10 baseline comparisons + three types of ablations (divergence/solver/width) + toy experiment + category-wise reporting + failure case analysis + rich supplementary experiments in the appendix covering MVTec AD, multi-class AD, DINOv3 backbones, etc., making it extremely solid.
Writing Quality: ⭐⭐⭐⭐⭐ Clear motivation (from RealNVP limitations to FM mapping, with toy example visualization), complete method description (architecture diagrams + step-by-step formulas), logically sound experiments (main results $\rightarrow$ ablations $\rightarrow$ analysis $\rightarrow$ failure cases), and extensive supplementary materials.
Value: ⭐⭐⭐⭐ Delivers a practical real-time solution for multi-view anomaly detection (18.77 FPS on a consumer GPU), matching SOTA on core benchmarks. The finding that the divergence term can be omitted provides valuable guidance for subsequent applications of Flow Matching in anomaly detection. However, scenarios where multi-view clearly outperforms single-view remain relatively limited, and the actual industrial deployment value remains to be verified.