AnyMatch: Supercharging Universal Multi-Modal Image Matching with Large-Scale Single-View Images¶

[ECCV 2026][3D Vision][Multimodal Image Matching] AnyMatch proposes a framework for synthesizing multimodal, multiview matching data from vast collections of single-view images at zero cost. Monocular depth estimation + 3D reprojection + diffusion inpainting enable geometrically consistent viewpoint transformations, while cross-modal image translation generates four modalities—infrared/depth/normals/events—and sample-level geometric consistency verification (SGCV) filters generation artifacts; after fine-tuning LoFTR/EDM/RoMa on the synthetic Any-syn dataset, cross-modal matching performance substantially surpasses existing synthesis methods such as MINIMA, with strong zero-shot generalization to unseen medical/remote-sensing modalities.

Tags: ECCV 2026 · 3D Vision · Multimodal Image Matching · Synthetic Data Generation · 3D Geometric Consistency · Viewpoint Transformation · Modality Transformation

Conference: ECCV 2026
arXiv: 2606.31077
Code: None
Area: 3D Vision
Keywords: Multimodal Image Matching, Synthetic Data Generation, 3D Geometric Consistency, Viewpoint Transformation, Modality Transformation

TL;DR¶

AnyMatch proposes a framework for synthesizing multimodal, multiview matching data from vast collections of single-view images at zero cost. Monocular depth estimation + 3D reprojection + diffusion inpainting enable geometrically consistent viewpoint transformations, while cross-modal image translation generates four modalities—infrared/depth/normals/events—and sample-level geometric consistency verification (SGCV) filters generation artifacts; after fine-tuning LoFTR/EDM/RoMa on the synthetic Any-syn dataset, cross-modal matching performance substantially surpasses existing synthesis methods such as MINIMA, with strong zero-shot generalization to unseen medical/remote-sensing modalities.

Background & Motivation¶

Multimodal image matching (e.g., RGB-infrared, RGB-depth, and RGB-event camera) underpins visual localization and multisensor fusion, but its central bottleneck is the lack of large-scale training data with precise geometric annotations. Collecting real data requires expensive synchronized multisensor hardware and precise calibration, making it extremely costly. Existing real datasets (e.g., MegaDepth) also rely on the SfM-MVS pipeline to generate geometric ground truth, which is applicable only to static scenes with high photometric consistency and has inherent limitations: restricted scene diversity, accumulated reconstruction errors, and an inability to handle multimodal images.

Existing synthetic data methods each have shortcomings: image translation methods (e.g., MINIMA) translate modalities on existing multiview RGB datasets and directly inherit SfM-MVS geometric annotations, making errors inevitable and limiting scene coverage to that of the source datasets; 2D homography methods simulate geometric deformation only in the image plane, discarding cross-view 3D geometric consistency and failing to generate physically plausible occlusion patterns; game-engine methods struggle to achieve photorealism, with a substantial domain gap particularly in modeling material reflections and global illumination.

The tension is that the internet contains vast collections of single-view images (millions in datasets such as GLDv2, SA-1B, and COCO), which naturally cover diverse scene content and long-tail distributions and could, in principle, provide unprecedented generalization potential for matching models—but an effective mechanism is missing for converting them into multimodal, multiview training data with precise 3D geometric supervision. Core Idea: combine monocular depth estimation, 3D reprojection, diffusion inpainting, and cross-modal translation into a complete pipeline that starts from any single image and automatically generates strictly geometrically consistent multimodal, multiview matching pairs and their pixel-level annotations.

Method¶

Overall Architecture¶

AnyMatch addresses the following core problem: given an arbitrary single-view RGB image, how can a set of multimodal, multiview image pairs be generated together with pixel-level correspondence annotations that strictly satisfy 3D geometric consistency? The framework is decoupled into two independent, composable core transformations—viewpoint transformation and modality transformation. The two branches run in parallel and are then combined, followed by geometric consistency verification to select high-quality samples.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Input: single-view RGB image<br/>512×512, from GLDv2/SA-1B"] --> B["Monocular depth estimation<br/>Moge V1 + random affine perturbation"]
    B --> C["3D point cloud reconstruction<br/>Random intrinsics K → lift pixels into 3D"]
    C --> D["Novel-view rendering<br/>Random extrinsics [R|T] → differentiable rasterization"]
    D --> E["Diffusion inpainting<br/>SD v2 fills occluded regions"]
    A --> F["Modality Transformation<br/>IR / depth / normals / events generated in parallel"]
    E --> G["Viewpoint × modality combination<br/>Inherit 3D geometric GT (K, [R|T], D, D')"]
    F --> G
    G --> H["SGCV geometric consistency verification<br/>PCK@5 ≥ η → retain, otherwise discard"]
    H --> I["Output: multimodal multiview matching pairs<br/>+ pixel-level correspondence annotations"]

The input is a 512x512 single-view RGB image \(I_{\text{sin}} \in T_{\text{sin}}\) sampled from a public dataset. The viewpoint transformation branch first uses a pretrained monocular relative-depth model \(M_{\text{rd}}\) (Moge V1) to predict a dense relative depth map \(D_r\). After random affine perturbation to simulate depth-scale uncertainty, randomly generated intrinsics \(K\) are used to lift 2D pixels into a 3D point cloud \(P\); randomly sampled extrinsics \([R|T]\) then transform the point cloud into a new viewpoint coordinate system \(P' = R \cdot P + T\). Differentiable mesh rendering (point cloud → triangle mesh → perspective projection → barycentric interpolation) produces an initial novel view \(I_{\text{novel}}\) and an occlusion mask \(M_{\text{occ}}\); finally, Stable Diffusion v2 performs context-aware inpainting of occluded regions to obtain \(I_{\text{inp}} = M_{\text{inpaint}}(I_{\text{novel}}, M_{\text{occ}})\).

In parallel, the modality transformation branch performs cross-modal translation on the original image \(I_{\text{sin}}\), generating four target-modality images \(I_{\text{modal}} = F(I_{\text{sin}})\): infrared (DiffV2IR, an LDM fine-tuned with two-stage LoRA), depth (Moge V1), normals (Moge V2), and events (event streams synthesized through motion simulation). Since modality transformation preserves geometric structure, combining the outputs of the two branches allows the camera parameters \(\{K, [R|T]\}\) and depth maps \(\{D, D'\}\) produced during viewpoint transformation to be directly inherited as geometric ground truth, without separate annotations for each modality. Finally, SGCV uses known geometric ground truth to cross-check matching quality, filtering low-quality samples with geometric inconsistencies caused by inpainting hallucinations. Only samples satisfying \(\text{PCK}@\tau \geq \eta\) enter the final training dataset, Any-syn.

Key Designs¶

1. Viewpoint Transformation: 3D geometrically consistent generation from a single view to multiple views

The central limitation of existing synthetic data methods is their inability to provide accurate 3D geometric ground truth for multiview image pairs—SfM-MVS accumulates errors, while homographies lack true 3D geometry. AnyMatch takes a "lift, then project" approach: a monocular depth model predicts relative depth \(D_r\), followed by random affine perturbation

\[D = 1/(\alpha \cdot (1/D_r) + \beta + \epsilon)\]

to simulate depth-scale uncertainty, where \(\alpha \in (0.5, 2.0)\) corresponds to scale drift caused by focal-length changes, \(\beta \in (0, 0.3)\) corresponds to systematic sensor errors and nonzero-mean noise, and \(\epsilon\) is a numerical stability term. This perturbation is designed to teach downstream matching models to tolerate imprecise depth estimates during training. Randomly generated intrinsics \(K\) (focal length \(f \in (0.58, 0.88)\), with the principal point fixed at the normalized image center \((0.5, 0.5)\)) then lift each pixel \((u_i, v_i)\) into 3D coordinates:

\[P = \begin{bmatrix}X \\ Y \\ Z\end{bmatrix} = D(u_i, v_i) \cdot K^{-1}\begin{bmatrix}u_i \\ v_i \\ 1\end{bmatrix}\]

Novel-view generation is controlled through random extrinsics \([R|T]\): rotation angles are sampled within \([-7.5^\circ, +7.5^\circ]\), translation components \(t_x, t_y\) within \([-0.3, +0.3]\), and \(t_z\) within \([-0.5, +0.5]\). Adjusting the range of extrinsic variation controls data difficulty and supports curriculum learning. After transforming the point cloud into the new coordinate system, differentiable mesh rendering (detailed in the appendix: point cloud → triangle mesh → perspective projection matrix \(\mathbf{\Pi}\) → NDC space → barycentric interpolation) generates an initial novel view \(I_{\text{novel}}\) and an occlusion mask \(M_{\text{occ}}\), while the Z coordinates of \(P'\) directly form the corresponding depth map \(D'\). Finally, diffusion inpainting fills the occluded regions, \(I_{\text{inp}} = M_{\text{inpaint}}(I_{\text{novel}}, M_{\text{occ}})\). The core advantage of viewpoint transformation is that all geometric annotations (camera parameters, depth maps, and pixel correspondences) are computed through explicit 3D reprojection, strictly satisfying epipolar geometry constraints and entirely bypassing error accumulation in SfM-MVS.

2. Modality Transformation: parallel cross-modal translation from one modality to multiple modalities

Modality transformation addresses how to expose matching models to appearance differences between sensors. AnyMatch uses four parallel pipelines to translate the original RGB image into four target modalities. The central principle is to preserve geometric structure (without introducing deformation), ensuring that geometric ground truth can be directly reused when combined with viewpoint transformation.

Infrared (IR): the two-stage diffusion model DiffV2IR is used. In the first stage, an LDM is fine-tuned with LoRA on a large infrared image collection, using the unified text prompt "an infrared image" to establish a semantic association with the infrared concept; in the second stage, RGB-conditioned fine-tuning on paired RGB-IR data teaches the model to synthesize outputs that reflect infrared physical properties (thermal radiation patterns) from visible-light inputs.
Depth and normals: pretrained monocular models Moge V1 (relative depth estimation) and Moge V2 (surface normal estimation) are used directly, without additional training. Their outputs serve as pseudo-depth and pseudo-normal maps.
Events: event streams are synthesized through motion simulation. Each event-camera pixel independently monitors changes in log brightness \(L = \log(I_{\text{sin}})\). When the brightness change \(\Delta L(\mathbf{x}_k, t_k)\) exceeds the contrast threshold \(C\), an event \(e_k = (\mathbf{x}_k, t_k, p_k)\) is triggered, with \(p_k = \pm 1\) indicating brightening/darkening polarity. The threshold \(C \in [0.05, 0.5]\) and polarity are randomly set to simulate different sensor characteristics, while small random motions are applied to the input frame to simulate real camera motion.

The result is \(I_{\text{modal}} = F(I_{\text{sin}})\), where \(F\) is the modality transformation module. The four pipelines operate entirely in parallel and independently. Adding a target modality only requires its corresponding translation pipeline, without changes to viewpoint transformation or geometric annotations.

3. Sample-level geometric consistency verification (SGCV): cross-checking data quality with GT

Although diffusion inpainting can produce visually coherent content, it may "hallucinate" spurious textures that violate the original scene's geometric and semantic structure. These artifacts can mislead matching models into learning incorrect feature correspondences. SGCV screens single-modality image pairs (\(I_{\text{sin}}\) and \(I_{\text{inp}}\)) before modality transformation, using known geometric ground truth to cross-check whether inpainting has deviated from the true geometry.

Specifically, the high-performance dense matching model RoMa first extracts a dense correspondence set \(\mathcal{M}_{\text{init}} = \{(x_{\text{sin}}^i, x_{\text{inp}}^i)\}_{i=1}^N\) between the original image and the inpainted novel view. Known camera parameters and depth maps are then used to compute the GT corresponding position \(\hat{x}_{\text{inp}}^i\) for each matched point through 3D reprojection, yielding the endpoint error:

\[\text{EPE}(x_{\text{sin}}^i) = \|x_{\text{inp}}^i - \hat{x}_{\text{inp}}^i\|^2\]

Finally, the proportion of correct correspondences \(\text{PCK}@\tau\) (\(\tau = 5\) pixels) is computed:

\[\text{PCK}@\tau = \frac{|\{(x_{\text{sin}}^i, x_{\text{inp}}^i)\}_{i=1}^N \mid \text{EPE}(x_{\text{sin}}^i) < \tau|}{N} \times 100\%\]

A sample is retained when \(\text{PCK}@\tau \geq \eta\) (\(\eta = 0.6\)), and discarded otherwise. The elegance of this method lies in "using the pipeline's strength to compensate for its weakness": 3D reprojection provides precise pixel-correspondence GT, which is then used to detect geometric shifts caused by diffusion inpainting. Ablation results identify this as the most important quality-control mechanism: removing SGCV reduces @10° from 35.80 to 31.24.

A Worked Example¶

Consider a building-facade image from GLDv2 to walk through the full AnyMatch pipeline. The input is a 512x512 RGB image \(I_{\text{sin}}\). Moge V1 first predicts a relative depth map \(D_r\), and affine perturbation (sampling \(\alpha=1.3\), \(\beta=0.12\)) produces the transformed depth \(D\). Randomly generated intrinsics (\(f=0.72\)) and extrinsics (rotation of 4.2 degrees, translation \([0.15, -0.12, 0.25]\)) then lift 2D pixels into approximately 260,000 3D points → transform them into the new viewpoint coordinate system → render them through differentiable rasterization to generate \(I_{\text{novel}}\). At this point, approximately 15% of pixels are marked as holes due to occlusion (the corresponding positions in \(M_{\text{occ}}\) are 0). Stable Diffusion v2 performs context-aware inpainting on these holes to obtain \(I_{\text{inp}}\).

Meanwhile, the modality transformation branch processes \(I_{\text{sin}}\) in parallel: DiffV2IR synthesizes an infrared image (building-window regions exhibit different thermal radiation characteristics due to temperature differences), Moge V1/V2 directly output depth and normal maps, and motion simulation generates an event stream with a random threshold \(C=0.2\). Each modality image \(I_{\text{modal}}\) is paired with \(I_{\text{inp}}\), directly inheriting the camera parameters \(\{K, [R|T]\}\) and depth maps \(\{D, D'\}\) produced during viewpoint transformation as pixel-level GT annotations.

Finally, SGCV performs verification: RoMa extracts approximately 2000 dense correspondences between \(I_{\text{sin}}\) and \(I_{\text{inp}}\). Known geometric parameters are used to compute the EPE of each pair, yielding \(\text{PCK}@5 = 72\%\), above the threshold \(\eta=0.6\). This image group passes the quality check, and all modality pairs are retained. The final output comprises 4 cross-modal matching pairs (RGB-IR, RGB-Depth, RGB-Normal, RGB-Event), each accompanied by precise pixel-level geometric ground truth for direct use in matching-model training.

Loss & Training¶

AnyMatch itself is a data generation framework with no trainable parameters. For downstream use, pretrained matching models are fine-tuned on Any-syn. AnyMatch constructs 500,000 training pairs from GLDv2 (covering five modality combinations: RGB-RGB/IR/Depth/Normal/Event) and 10,000 test pairs from SA-1B. Following the MINIMA setup, only RGB-IR, RGB-Depth, and RGB-Normal pairs are used for training; RGB-Event and RGB-RGB are used only for testing. During training, dense matching supervision is applied uniformly to all pixels, including inpainted regions.

Fine-tuning configurations for the three backbones: (1) EDM retains its original learning rate of \(1 \times 10^{-4}\) and is fine-tuned for 10 epochs with batch size 4; (2) LoFTR uses a learning rate of \(8 \times 10^{-4}\) and is fine-tuned for 10 epochs with batch size 4; (3) RoMa uses encoder/decoder learning rates of \(6.25 \times 10^{-8}\) and \(1.25 \times 10^{-6}\), respectively, converging in just 4 epochs with batch size 2. All models are trained on 4 NVIDIA RTX 4090 GPUs. Pose estimation uses consistent RANSAC hyperparameters to ensure fair comparisons.

Key Experimental Results¶

Main Results¶

Pose estimation on the test subset of the synthetic Any-syn dataset (AUC of pose error; selected comparisons among the original, MINIMA-fine-tuned, and AnyMatch-fine-tuned versions of the EDM/LoFTR/RoMa backbones).

Method	RGB-IR @5°	RGB-IR @10°	RGB-IR @20°	RGB-Depth @10°	RGB-Normal @10°	RGB-Event @10°
EDM (original)	9.20	19.63	32.78	0.41	14.81	0.83
EDM_MINIMA	14.70	31.56	51.20	11.25	25.99	3.30
EDM_AnyMatch	28.80	50.00	68.53	20.36	36.65	9.32
LoFTR (original)	8.35	17.92	30.13	0.17	8.70	1.08
LoFTR_MINIMA	13.09	28.75	48.20	9.27	22.59	3.43
LoFTR_AnyMatch	22.30	41.45	60.91	9.78	26.61	3.94
RoMa (original)	30.08	48.65	65.94	7.00	36.24	3.10
RoMa_MINIMA	40.00	60.62	76.27	22.16	49.33	7.29
RoMa_AnyMatch	50.91	69.88	82.75	29.39	56.50	11.87

RoMa_AnyMatch reaches 69.88% on RGB-IR @10°, an absolute gain of 9.26 percentage points over RoMa_MINIMA (60.62%) and a relative gain of 43.6% over the original RoMa (48.65%). RGB-Event has an especially large modality gap because event cameras sense asynchronous brightness changes, so all methods perform substantially worse than on other modalities. Nevertheless, EDM_AnyMatch achieves 9.32% at @10°, far above EDM_MINIMA's 3.30% (+6.02 percentage points).

Evaluation on real datasets (METU-VisTIR RGB-IR pose estimation + DIODE RGB-Depth/Normal + DSEC RGB-Event homography estimation; selected core metrics).

Method	RGB-IR @10°	RGB-Depth @5px	RGB-Normal @5px	RGB-Event @10px
EDM (original)	15.82	5.75	18.25	8.89
EDM_MINIMA	32.80	24.89	33.72	13.63
EDM_AnyMatch	35.80	28.75	40.04	13.57
RoMa (original)	48.12	29.47	46.01	8.72
RoMa_MINIMA	60.70	51.09	59.24	13.16
RoMa_AnyMatch	62.61	49.83	60.02	14.25

In zero-shot evaluation on unseen modalities (the MMIM dataset, covering 13 medical/remote-sensing cross-modal combinations), AnyMatch has a particularly pronounced advantage in remote sensing—EDM_AnyMatch reaches 38.69 at remote-sensing @5px, far above EDM_MINIMA's 30.52 and the original EDM's 26.50; LoFTR_AnyMatch reaches 39.76 at remote-sensing @5px, far above LoFTR_MINIMA's 33.96. This indicates that models trained with AnyMatch learn modality-independent, geometry-aware feature representations rather than merely fitting the appearance distributions of known modalities.

Ablation Study¶

Ablations on METU-VisTIR RGB-IR with EDM as the baseline (AUC of pose error).

Config	@5°	@10°	@20°	Note
1 modality w/o SGCV	13.55	27.71	44.50	RGB-IR only, without geometric verification
3 modalities w/o SGCV	15.63	31.24	48.29	Joint training on RGB-IR/Depth/Normal
3 modalities w/ SGCV (η=0.6)	17.13	35.80	55.24	Full AnyMatch configuration
3 modalities + from scratch	5.69	14.16	27.45	Training from scratch without pretrained weights
3 modalities + homography	15.45	30.78	46.16	Use a 2D homography instead of 3D viewpoint transformation

SGCV threshold ablation (η from 0.7 to 0, i.e., no filtering):

η	0.7	0.6	0.5	0.4	0.3	0 (no filtering)
AUC@10°	32.52	35.80	31.47	32.04	31.32	31.24

Key Findings¶

Joint multimodal training yields substantial gains: training with three modalities rather than one improves @20° from 44.50 to 48.29 (+3.79), indicating that cross-modal collaborative learning helps extract robust, modality-independent features.
SGCV is the most important quality-control mechanism: adding SGCV improves @10° from 31.24 to 35.80 (+4.56), peaking at η=0.6. Overly strict filtering (η=0.7) discards useful samples, while overly permissive filtering fails to eliminate hallucination artifacts—there is an optimal balance.
3D viewpoint transformation substantially outperforms 2D homographies: the homography variant achieves only 30.78 at @10° vs. AnyMatch's 35.80, confirming that parallax and occlusion relationships generated through 3D reprojection better satisfy real physical constraints.
Pretraining priors are indispensable: training from scratch reaches only approximately 50% of the fine-tuned model's @20° performance (27.45 vs 55.24), indicating that general matching priors in pretrained matching models are critical for cross-modal adaptation.
LoFTR is more sensitive to synthetic data quality: appendix analysis shows that on low-quality synthetic pairs (PCK@5 ∈ [0.2, 0.4]), LoFTR_AnyMatch's @20° drops by 30.94% relative to the high-quality group, while EDM_AnyMatch drops by only 10.84%, indicating that Transformer-based LoFTR is more vulnerable to geometrically inconsistent artifacts.

Highlights & Insights¶

Decoupled "single view → multimodal multiview" design: AnyMatch's most elegant feature is the complete decoupling of viewpoint and modality transformations—the two branches run independently and are then combined, with geometric annotations generated solely by viewpoint transformation and reused by modality transformation at zero cost. Adding any new modality requires only one additional translation pipeline, leaving the rest unchanged. This factorization of "geometry generation × appearance translation" can extend to other tasks requiring cross-modal paired data.
SGCV is a self-checking mechanism that "uses strengths to compensate for weaknesses": since 3D reprojection provides precise pixel-correspondence GT, that GT can be used to detect whether diffusion inpainting has hallucinated content that deviates from the true geometry. This design pattern uses a pipeline's own strengths to monitor its weaknesses and can transfer to any "generate + filter" data synthesis pipeline.
Affine depth perturbation as implicit data augmentation: by applying random perturbations \(D = 1/(\alpha \cdot (1/D_r) + \beta + \epsilon)\) to monocular depth, AnyMatch exposes downstream matching models to supervision with depth noise during training, teaching them inherent robustness to inaccurate depth estimates—a low-cost, effective way to "teach models to tolerate input noise."
Controllable difficulty supports curriculum learning: adjusting the range of extrinsic variation (rotation amplitude and translation magnitude) systematically controls the severity of viewpoint changes in generated data, allowing training data to be organized from easy to hard and naturally supporting curriculum learning.

Limitations & Future Work¶

Inpainting hallucinations are the central bottleneck: diffusion inpainting may generate spurious content that violates geometric and semantic consistency. SGCV can only filter out bad samples, not repair them. The substantial degradation of LoFTR on low-quality synthetic pairs in the appendix corroborates this problem: the quality floor of synthetic data determines the performance ceiling of some artifact-sensitive backbones.
Dependence on monocular depth estimation: all 3D geometric consistency rests on monocular depth predictions. Although random perturbations mitigate scale ambiguity and systematic bias, depth errors propagate directly into geometric annotations in difficult regions such as depth discontinuities (object boundaries) and transparent/specular surfaces.
Some real-dataset metrics fall below MINIMA: RoMa_AnyMatch achieves 49.83 on RGB-Depth @5px, slightly below RoMa_MINIMA's 51.09; LoFTR_AnyMatch achieves 25.83 on RGB-IR @10°, below LoFTR_MINIMA's 30.84. The appendix analyzes LoFTR's degradation: LoFTR is more sensitive to hallucinations/depth errors in low-quality synthetic pairs. This also reflects a remaining domain gap between AnyMatch's synthetic data and real images, as texture and illumination distributions in inpainted regions may differ from those of real scenes.
Directions for improvement: (1) replace Moge V1 with a stronger depth estimation model to reduce the propagation of depth errors at their source; (2) incorporate geometric constraints (such as an epipolar geometry consistency loss) into inpainting so that the network pursues geometric correctness as well as visual coherence; (3) use SGCV's PCK scores for soft-weighted training (higher weights for higher-quality samples) rather than a simple hard-threshold binary decision; (4) mix AnyMatch and MINIMA data during training to exploit the complementarity of the two synthesis approaches and cover a broader data distribution.

vs MINIMA: MINIMA translates modalities on existing multiview RGB datasets and directly inherits SfM-MVS geometric annotations; AnyMatch generates its own 3D geometry from single views, avoiding limits imposed by existing multiview datasets' scene coverage and SfM error accumulation in annotations. AnyMatch has a clear advantage in zero-shot generalization to unseen modalities, but MINIMA remains competitive on some known modalities, indicating complementary distributions between the two data-generation approaches.
vs 2D homography methods (CrossHomo, etc.): homographies simulate geometric deformation only in the image plane, without producing true depth-dependent occlusions and parallax; AnyMatch's 3D reprojection explicitly models the physical relationship from depth → parallax, and the ablation (35.80 vs 30.78 at @10°) directly confirms the benefit of 3D geometric consistency for matching-model training.
vs game-engine methods (Kubric, etc.): game engines precisely control scene parameters and sensor models but are limited in rendering realism; starting from real photographs, AnyMatch naturally offers photorealistic textures and illumination, at the cost of precise scene control. There is an inherent trade-off between "realism" and "controllability."
Insights: AnyMatch's factorized "geometry generation × appearance translation" paradigm can extend to other domains requiring cross-domain paired data—for example, automatically converting RGB object detection/segmentation annotations into infrared or depth-modality annotations, or generating multiview, multi-illumination supervision for NeRF/3DGS training.

Rating¶

Novelty: ⭐⭐⭐⭐⭐ The first complete framework to generate multimodal, multiview matching data from arbitrary single-view images at zero cost; its viewpoint-modality decoupling is simple and powerful, and SGCV is a practical, elegant self-checking mechanism.
Experimental Thoroughness: ⭐⭐⭐⭐⭐ Extensive evaluation across 3 matching backbones, 4 modality combinations, and 5 datasets (synthetic + real, in-domain + zero-shot). Ablations cover modality count, SGCV thresholds, training strategies, 2D vs 3D, and camera-parameter sensitivity; the appendix adds RoMa ablations and analysis of LoFTR degradation.
Writing Quality: ⭐⭐⭐⭐ Clear method descriptions, well-integrated equations and illustrations, and a substantial appendix. Some ablation analyses could go deeper (e.g., the lack of a theoretical explanation for why η=0.6 is optimal), and placing LoFTR's degradation analysis in the appendix rather than the main text seems somewhat evasive.
Value: ⭐⭐⭐⭐⭐ Addresses the most fundamental data bottleneck in multimodal matching with a simple, scalable pipeline. Any new modality can be incorporated by adding a single translation pipeline, offering substantial practical value for advancing universal multimodal matching models.