MATCH: Flow Matching for Multi-View Anomaly Detection¶

Conference: ECCV 2026
arXiv: 2606.24375
Code: https://github.com/m-kruse98/MATCH
Area: Anomaly Detection / Multi-View Vision
Keywords: Multi-view anomaly detection, Flow Matching, normalizing flows, density estimation, industrial defect detection

TL;DR¶

MATCH is the first Flow Matching-based multi-view anomaly detection method. It performs density estimation on pre-trained feature spaces using continuous normalizing flows in the form of ODEs. By omitting the divergence term, it achieves real-time inference (18.77 FPS) and obtains SOTA detection and segmentation performance on both the Real-IAD and MANTA-Tiny benchmarks.

Background & Motivation¶

Background: The mainstream paradigm of industrial anomaly detection is semi-supervised learning—using only normal samples during training, and detecting anomalies that deviate from the normal distribution during inference. Existing high-performance methods extensively use RealNVP-based normalizing flows for likelihood estimation, such as DifferNet, CS-Flow, FastFlow, CFlow, and Multi-Flow.

Limitations of Prior Work: The RealNVP architecture has two fundamental limitations. First, its coupling block structure requires the network to be invertible and have an easily computable Jacobian determinant, which severely constrains the representational capacity of the network architecture, restricting it to stacking a limited number of layers. Second, in multi-view scenarios, the image data is highly dimensional (concatenation of feature maps from multiple views), and RealNVP is prone to failure when modeling high-dimensional distributions—which the paper illustrates intuitively through a "Two Moons" toy experiment: when 2D moons data is embedded into a \(d\)-dimensional space, Flow Matching correctly captures the distribution across various dimensions, whereas RealNVP begins to produce degenerate density estimations (negative BPD) when \(d>2\).

Key Challenge: Multi-view anomaly detection requires capturing minute deviations (anomalies) in extremely high-dimensional feature spaces, but the representational capacity of the RealNVP architecture is restricted by its invertibility constraints, making it incapable of accurately modeling high-dimensional distributions. Meanwhile, precise likelihood estimation requires calculating divergence terms, which significantly increases the inference cost, hindering real-time industrial deployment.

Goal: (1) Replace RealNVP with Flow Matching for density estimation in multi-view anomaly detection; (2) omit the divergence term to improve inference speed while maintaining detection accuracy; (3) perform a comprehensive benchmark test on the MANTA-Tiny dataset for the first time.

Key Insight: As a modern training paradigm for continuous normalizing flows (CNF), Flow Matching does not require the network to be invertible and can use any general architecture (such as ResNet or Transformer), making it naturally suited for high-dimensional data modeling. Its ODE form supports both the generative direction (noise \(\rightarrow\) data) and the density estimation direction (data \(\rightarrow\) Gaussian space), the latter of which can be utilized to calculate anomaly scores.

Core Idea: Map multi-view features to a standard Gaussian space using the ODE of Flow Matching, utilize the negative log-likelihood of the sample in the Gaussian space (essentially the Mahalanobis distance) as the anomaly score, and omit the divergence term to accelerate inference by more than 3x without loss of accuracy.

Method¶

Overall Architecture¶

The overall pipeline of MATCH consists of "frozen feature extraction \(\rightarrow\) multi-scale feature alignment \(\rightarrow\) Flow Matching latent space mapping \(\rightarrow\) reverse ODE integration to compute anomaly scores". During training, normal multi-view images are input and processed by a frozen WideResNet50 to extract three layers of feature maps. After incorporating the timestep \(t\) and view \(v\) embeddings, a bottleneck network aligns and fuses the multi-scale features. Then, a three-layer decoder restores them to the original dimensions of each scale, and the network is trained with the OT-CFM loss to learn the vector field mapping the data distribution to the standard Gaussian. During inference, reverse ODE integration is run to integrate the test samples from \(t=1\) to \(t=0\), obtaining their mapping \(x_0\) in the latent Gaussian space. \(S(x) = \|x_0\|^2/2 + (d/2)\log(2\pi)\) is used as the anomaly score, generating multi-scale anomaly maps for segmentation and detection.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Multi-view Normal Images<br/>256×256"] --> B["Frozen WideResNet50<br/>Extract 3 feature map layers"]
    B --> C["Bottleneck: Spatial Alignment<br/>+ Concatenation + ResNet Bottleneck Blocks"]
    C --> D["Add timestep t and view v embeddings"]
    D --> E["Three-layer Decoder Upsamples step-by-step<br/>Restoring features of each scale"]
    E --> F["OT-CFM Loss Training<br/>Learn Data→Gaussian Vector Field"]
    F --> G["Reverse ODE Integration<br/>Compute latent space x_0"]
    G --> H["S(x) = ‖x_0‖²/2<br/>Anomaly Maps → Detection/Segmentation"]

Key Designs¶

1. Flow Matching Replacing RealNVP: Fundamentally Relieving Architectural Constraints

RealNVP's coupling blocks require every layer to be invertible and have an easily computable Jacobian, which restricts the network to specific structures (e.g., affine coupling layers) and limits the depth due to representation capacity saturation. Flow Matching is completely different—it learns a time-dependent vector field \(u_t(x)\), defining a continuous transformation from noise \(p_0\) to data \(q\) via the ODE \(d(\phi_t)/dt = u_t(\phi_t)\), and imposes no invertibility or Jacobian constraints on the network architecture. During training, the OT-CFM loss is used: \(L = E[\|u_t^\theta(x_t) - (x_1 - x_0)\|^2]\), where \(x_t = (1-t)x_0 + t*x_1\) is the linearly interpolated intermediate point. This loss is a simple regression target (predicting the straight-line direction from the current point to the target data point) and does not require backpropagation through the ODE solver. Once trained, forward integration (\(t=0 \rightarrow 1\)) is used for generation, and reverse integration (\(t=1 \rightarrow 0\)) is used for density estimation. This architectural flexibility allows MATCH to use ResNet-style bottlenecks and decoders, giving freedom in designing depth and width. Experiments show that the optimal width is 768, far exceeding the typical configuration of RealNVP.

2. Multi-Scale Latent Space Density Estimation and Reverse Distillation Architecture

Directly applying Flow Matching in pixel space has been proven ineffective for anomaly detection, as the differences between normal and abnormal images are too subtle at the pixel level. MATCH learns distributions in the latent space of a pre-trained WideResNet50, extracting the first three layer feature maps \(x_1^i \sim q^i\) (with sizes 256x64x64, 512x32x32, and 1024x16x16, respectively, for \(i=1,2,3\)). These feature maps are first spatially aligned via a bottleneck—the feature maps for \(i=1\) are downsampled with two layers of stride-2 convolutions, and those for \(i=2\) with one layer. They are all aligned to 16x16, concatenated, and further distilled through three ResNet-style bottleneck blocks (with group norm + ReLU) to represent the normal structure. The decoder consists of three layers \(D_i\) (containing 3, 4, and 6 decoder blocks, respectively), where each layer uses 1x1 conv + 3x3 stride-2 transposed convolutions to upsample, gradually restoring the spatial sizes of original scales to output three anomaly maps \(M_i\). This "compress-and-restore" Reverse Distillation structure forces the bottleneck to only preserve the common structure of normal samples, causing abnormal samples to yield larger reconstruction deviations during decoder restoration, thereby producing stronger responses on the anomaly maps. The timestep \(t\) (1024-dimensional sinusoidal positional encoding + 2-layer MLP) and view \(v\) (learnable embedding + MLP) are concatenated and injected into the bottleneck output and each decoder block, allowing the network to know the current flow path stage and the active view.

3. Omitting the Divergence Term: Exchanging "Incomplete Likelihood" for Real-Time Inference

The standard likelihood estimation formula for Flow Matching is \(\log p_1(x_1) = \log p_0(x_0) - \int \text{div}(u_t)(x_t) dt\), where the divergence term needs to be integrated along the ODE trajectory. In practice, the divergence is computed using the Hutchinson trace estimator: \(\text{div}(u_t)(x) = \mathbb{E}_z[z^T \cdot \partial_x u_t(x) \cdot z]\), which requires an extra backward pass per step. The authors found that on Real-IAD, the absolute value of the divergence term accounts for only 0.57% of the final score magnitude (\(|\text{div}|/|\log p_0(x_0)| = 0.0057\)), and adding the divergence (regardless of \(M=1,2,4\) sample vectors) has no impact on I-AUROC (91.172 vs 91.170). After omitting the divergence, the FPS surges from 5.40 (\(M=1\)) to 18.77, and the GPU memory drops from 10.28GB to 8.88GB. Therefore, the anomaly score in MATCH is simplified to \(S(x) = \|x_0\|^2/2 + (d/2)\log(2\pi)\). Omitting the constant term, this is essentially the squared Mahalanobis distance from \(x_0\) to the standard Gaussian—the further \(x_0\) is from the center, the more likely it is an anomaly. The authors hypothesize that this is because the learned vector field is highly smooth, leaving local divergence close to zero.

4. Multi-Granularity Anomaly Score Fusion: Local and Global Complementarity

MATCH outputs three anomaly maps \(M_i\) from the three decoder layers, each obtained by calculating \(S(x)\) along the feature dimension. For the segmentation task, these maps are directly summed: \(M_{\text{segment}} = \sum(M_i)\), and then bilinearly interpolated to the original image size. The low-level \(M_1\) is adept at capturing minute, local low-level anomalies (such as scratches), while the high-level \(M_2\) and \(M_3\) are more sensitive to high-level structural anomalies; stacking the three maps achieves multi-granularity coverage. For image-level detection, only \(M_{\text{detect}} = M_2 + M_3\) is used (discarding \(M_1\)), as prior work has established that high-level representations are better suited for generating a single image-level anomaly score, while low-level deviations are easily triggered by normal texture variations, introducing noise. The image-level score is taken as the maximum value of \(M_{\text{detect}}\). For object-level (multi-view) detection, the maximum of the image-level scores across all views of the object is taken—if any view presents an anomaly, the entire object is classified as anomalous.

Loss & Training¶

The training objective is the OT-CFM loss (formula shown in Key Designs 1), which is summed after computing losses on the three feature scales respectively. The optimizer is AdamW (\(\text{lr}=1\text{e-}4\), \(\beta=(0.9, 0.95)\), weight decay \(= 0.01\)), requiring no gradient clipping or scaling (unlike RealNVP). All images are resized to 256x256 with a batch size of 8, and trained for 150 epochs. During inference, the Euler method is used to solve the ODE with a step size of 0.2, totaling 5 forward passes. The ODESolver ablation study demonstrates that the Euler method is not only the fastest (13.13 FPS) but also slightly outperforms Midpoint (91.08) and RK4 (91.07) in terms of I-AUROC (91.17), indicating no accuracy-speed trade-off.

Key Experimental Results¶

Main Results¶

Real-IAD (30 industrial object classes, 5 views):

Metric	MATCH	Multi-Flow	RD4AD	EfficientAD	FastFlow
I-AUROC	91.17	90.27	89.26	77.26	79.26
S-AUROC	95.63	95.85	93.46	86.91	92.30
P-AUROC	99.24	96.47	98.81	89.64	90.18
P-AUPRO	94.76	87.91	92.70	71.31	66.99
P-AP	32.68	12.42	32.65	13.18	06.16

MATCH ranks first across all metrics except S-AUROC. Its P-AUPRO is 2.06 points higher than the runner-up RD4AD, and 6.85 points higher than Multi-Flow (which is also flow-based), verifying that Flow Matching is better suited for anomaly segmentation in high-dimensional feature spaces than RealNVP. S-AUROC is slightly lower than Multi-Flow (-0.22), which itself is specifically optimized for multi-view scenarios.

MANTA-Tiny (tiny objects, 5 major categories, multiple sub-categories):

Metric	MATCH	RD4AD	Multi-Flow	SimpleNet	EfficientAD
P-AUPRO	89.66	85.59	81.76	79.42	80.83
P-AUROC	95.65	94.60	93.07	93.63	92.21
I-AUROC	90.66	88.45	—	93.19	—

P-AUPRO leads the runner-up RD4AD by more than 4 percentage points. Broken down by category, Mechanics (industrial parts) achieves the highest performance (P-AUPRO 96.35), while Agriculture (natural items) is the lowest (75.52), owing to the high variance and ambiguous anomaly boundaries of natural categories. It falls slightly behind SimpleNet in I-AUROC, which the authors attribute to the difficulty of translating tiny defects in MANTA-Tiny into high image-level anomaly scores.

Ablation Study¶

Divergence Term Ablation (Real-IAD):

Method	M	I-AUROC	FPS	GPU Memory (GB)
MATCH (w/o Div)	—	91.172	18.77	8.88
Hutchinson	1	91.170	5.40	10.28
Hutchinson	2	91.169	3.16	10.70
Hutchinson	4	91.170	1.56	11.45
RQMC	1	91.170	—	—
RQMC	4	91.170	—	—

Incorporating the divergence (regardless of the estimation method and the number of sample vectors) yields zero improvement in detection accuracy, while the FPS drops by at least 3.5x. The divergence term accounts for only 0.57% of the final score, justifying its omission.

ODE Solver Ablation (Real-IAD):

Solver	Step Size	I-AUROC	FPS
Euler	0.20	91.17	18.77
Euler	0.10	91.12	6.54
Midpoint	0.20	91.08	6.55
RK4	0.20	91.07	2.48
Dopri5 (Adaptive)	—	91.06	0.48

The Euler method achieves a win-win in speed and accuracy with a large step size (0.2, requiring only 5 steps). More complex solvers do not improve the AUROC (all around 91.06) and suffer from significantly reduced speed.

Network Width Ablation: Incrementally increasing the feature dimensions of the bottleneck and decoder from 32 to 768 leads to a nearly linear rise in all metrics, peaking at a width of 768. Matching the parameter count of RD4AD to MATCH yields an I-AUROC of only 86.75 for RD4AD, confirming that the performance enhancement stems from the Flow Matching formulation rather than model capacity.

Key Findings¶

The divergence term can be completely omitted without loss of performance, which is key to making MATCH practical—implying that Flow Matching for anomaly detection does not require exact likelihood, but only relatively ranked anomaly scores.
Multi-scale anomaly map fusion is central to the massive lead in segmentation performance: low-level maps capture subtle texture defects, while high-level maps perceive structural anomalies, rendering them mutually complementary.
Performances on agricultural and grocery categories (Agriculture/Groceries) in MANTA-Tiny are significantly weaker than those in industrial and electronic categories (Mechanics/Electronics), as the former feature diverse normal sample morphologies and their anomaly evaluation relies more on semantic reasoning. The authors present several failure cases: withered defects in rice (only slight color differences), soybean cracks causing overall anomalies, and false positives caused by light reflections.
Evaluating the MATCH model trained on MANTA-Tiny directly on the complete MANTA test set shows no significant performance drop (P-AUPRO 89.23 vs 89.66), suggesting that MANTA-Tiny is a viable substitute benchmark for MANTA.

Highlights & Insights¶

The omission of the divergence term is the most clever design decision in this paper. Conventionally, using Flow Matching for density estimation requires full likelihood. However, the authors demonstrate experimentally that for tasks like anomaly detection that only require relative ranking, an "incomplete likelihood" (only the log density of the latent Gaussian) suffices, as the norm distributions of normal and abnormal samples in the latent space are already sufficiently separable. This effectively degrades expensive exact density estimation to an efficient distance-to-origin metric, presenting an elegant trade-off between theory and engineering.
Combining the Reverse Distillation architecture with Flow Matching is a natural yet overlooked combination. The "compress-and-restore" structure of RD is inherently suited for anomaly detection (with the bottleneck filtering out abnormal information), while Flow Matching provides it with architecture-unconstrained distribution modeling capabilities. Stacking the two leads to a massive lead over pure RealNVP schemes in segmentation metrics, suggesting that the choice of architectural paradigm and generative model are of equal importance.
The multi-view aggregation strategy is minimalist yet effective: taking the max for the image level and the max for the object level. This two-layer max operation introduces no extra parameters or training. This implies that in multi-view anomaly detection, the assumption that "an anomaly in any view makes the object anomalous" is strong enough, obviating the need for complex cross-view attention or fusion mechanisms.
The embedding mechanism for timestep and view information is highly reusable—injected into the bottleneck and each decoder block via sinusoidal positional encodings + MLPs, allowing the network to sense both the stage of the flow path and differences in feature distributions across views.

Limitations & Future Work¶

Insufficient handling of global structural anomalies: The authors acknowledge in the failure analysis that when multiple parts of an object each look normal individually but form an anomaly when combined (e.g., a capsule with blue caps on both ends instead of one blue and one white), MATCH's local anomaly maps fail to capture this global violation. This is because the segmentation branch prioritizes local textures, and while the detection branch has a global field of view, it lacks explicit modeling of "cross-local consistency". Incorporating cross-local attention or graph models might alleviate this.
Sensitivity to illumination and pose variations: Some false positives originate from light reflections or object rotations (illumination conditions unseen in the training set). This is because the model models the normal distribution only in 2D feature space and lacks 3D geometric priors to distinguish "changes in surface appearance due to viewpoint variations" from "actual surface defects".
S-AUROC falls behind the specialized multi-view method Multi-Flow: Although the gap is only 0.22 points, it suggests that a simple max-pooling aggregation might not be the optimal way to fuse object-level scores. Investigating complementary information between views (e.g., some defects are only visible in specific views and could be inferred logically from other views) could yield further improvements.
I-AUROC on MANTA-Tiny lags behind SimpleNet: SimpleNet performs supervised training by adding noise in the feature space to synthesize anomalies. This "synthetic anomaly supervision" might be superior to pure density estimation for detecting tiny defects. Future work could explore combining Flow Matching with synthetic anomalies.
Unexploited multi-view geometric constraints: The paper notes that future work will introduce explicit 3D constraints or multi-view correspondences (such as epipolar geometry) into the Flow Matching framework.
Only validated on ImageNet pre-trained WideResNet50: Although the supplementary material tests the DINOv3 backbone (with slightly reduced performance), larger modern backbones (such as DINOv2 ViT-L) or direct training in pixel space have not been explored.

vs Multi-Flow: Both are multi-view flow-based methods, but Multi-Flow is based on RealNVP (discrete normalizing flows) and is limited by the architectural constraints of coupling blocks. MATCH replaces it with Flow Matching, yielding higher expressivity and significantly better segmentation performance (P-AUPRO 94.76 vs 87.91). However, Multi-Flow performs slightly better in S-AUROC, likely due to its finer multi-view fusion strategy.
vs RD4AD: RD4AD utilizes a Reverse Distillation architecture for anomaly detection, but its teacher-student distillation does not involve explicit density estimation. MATCH repurposes its encoder-decoder structure but incorporates Flow Matching on the bottleneck to learn explicit probability distributions, achieving stronger anomaly localization capabilities. Parameter-controlled experiments confirm that the gains originate from Flow Matching rather than increased model capacity.
vs DifferNet / CS-Flow / CFlow: These single-view NF methods utilize RealNVP for density estimation and fail in multi-view scenarios. MATCH demonstrates that Flow Matching's distribution-fitting capability in high-dimensional feature spaces far exceeds that of RealNVP (in the same feature space, the RealNVP baseline's BPD is as high as 160.5, compared to only 1.83 for MATCH).
vs Novel-View Synthesis-based Multi-View Methods (e.g., SplatPose, PACLD): These methods synthesize a fixed 3D normal prototype (such as a 3DGS representation) to compare against new view renderings. The drawback is their inability to handle cases where the normal samples themselves exhibit high morphological variance. In contrast, MATCH's density estimation approach naturally supports learning multi-modal normal distributions.

Rating¶

Novelty: ⭐⭐⭐⭐ First to introduce Flow Matching into multi-view anomaly detection, and the discovery of omitting the divergence term holds practical value. However, the core architecture (RD + FM) is a combination of two known components, marking its methodological novelty as slightly above average.
Experimental Thoroughness: ⭐⭐⭐⭐⭐ Extensive evaluation on two large datasets + comparison with 10 baselines + three categories of ablations (divergence/solver/width) + toy experiment + category-by-category reporting + failure case analysis + appendix featuring MVTec AD, multi-class AD, and DINOv3 backbone results. Very solid.
Writing Quality: ⭐⭐⭐⭐⭐ Clear motivation (moving from the limitations of RealNVP to the adaptation of FM, with toy example visualizations), complete methodology description (architecture diagrams + step-by-step equations), rigorous experimental logic (main results \(\rightarrow\) ablations \(\rightarrow\) analysis \(\rightarrow\) failure cases), and rich supplementary materials.
Value: ⭐⭐⭐⭐ Delivers a practical real-time solution (18.77 FPS on a consumer GPU) for multi-view anomaly detection, establishing SOTA on core benchmarks. The discovery that the divergence term can be omitted provides valuable guidance for subsequent applications of Flow Matching in anomaly detection. However, scenarios where multi-view outperforms single-view remain relatively niche, and the actual industrial deployment value is yet to be fully validated.