LoMa: Local Feature Matching Revisited¶

Conference: ECCV2026
Paper: ECCV Paper
Code: https://github.com/davnords/LoMa
Area: 3D Vision
Keywords: local feature matching, data scaling, descriptor learning, visual localization, challenging matching evaluation

TL;DR¶

LoMa retains the basic division between the DaD detector, DeDoDe descriptor, and LightGlue matcher, but scales training data, updates the training recipe, and widens the matcher to reach 54.3 mAA@10px on the new HardMatch test set, 18.6 points above ALIKED+LightGlue.

Background & Motivation¶

Reconstructing a scene from unordered photographs requires identifying pixels that represent the same physical location across images. A conventional local feature pipeline first detects keypoints, then extracts descriptors, and finally establishes correspondences between the descriptor sets. Learned detectors and contextual matchers such as LightGlue improve robustness while preserving the ability to precompute each image's features and reuse them across image pairs. Recent prominent advances, however, have increasingly come from detector-free methods such as LoFTR and RoMa, and feed-forward reconstruction systems such as MASt3R and VGGT. This makes it tempting to conclude that selecting sparse keypoints first is a paradigm approaching its ceiling.

The paper challenges that causal interpretation rather than denying that sparse keypoints can miss corresponding regions. The authors observe that newer dense and reconstruction methods typically receive substantially more training data than traditional local matchers; with unequal data and compute, architecture alone cannot explain the gap. Meanwhile, common evaluations sample pairs from successfully reconstructed scenes, effectively excluding some difficult cases that older systems could not solve. As easy pairs become saturated, greater generalization may produce only a small change in the aggregate score, obscuring its practical importance. The authors therefore change both the training conditions and evaluation difficulty: the former tests how much local matching can still improve, while the latter makes those improvements observable.

LoMa is a scaled training recipe and model family, not an entirely new detector, descriptor, or attention architecture. HardMatch complements it with challenging cross-era, day-night, and extreme-viewpoint pairs, using manual correspondences to avoid depending entirely on successful existing reconstructions. Core Idea: retain a reusable sparse feature pipeline, train stronger descriptors and matchers with diverse geometric supervision, and evaluate genuinely difficult pairs instead of treating performance under older training conditions as the paradigm's ceiling.

Method¶

Overall Architecture¶

The input is an image pair, and the output is a set of keypoint correspondences for geometric estimation, SfM, or visual localization, rather than a directly predicted depth map or camera pose. Training proceeds through multi-source geometric supervision, descriptor pretraining, and matcher scaling; the trained descriptor remains frozen during matcher training. At inference time, DaD independently detects keypoints in each image, the descriptor extracts features at those locations, and the matcher enables interaction between the two feature sets. The matcher produces contextually refined descriptors, followed by mutual selection and confidence filtering. Dashed edges below indicate the use of trained models, not a requirement for ground-truth depth or training data at inference time.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    DATA["17 training datasets"] --> MIX["Multi-source<br/>geometric supervision"]
    MIX --> DESC["Descriptor pretraining"]
    DESC -->|Freeze descriptor| MATCH["Matcher scaling"]
    MIX -->|Training matches and matchability| MATCH
    INPUT["Image pair"] --> FEAT["DaD keypoints<br/>and independent descriptors"]
    DESC -.->|Trained feature extraction| FEAT
    FEAT --> PAIR["Cross-image matching"]
    MATCH -.->|Trained matching| PAIR
    PAIR --> OUTPUT["Mutual selection and filtering<br/>Keypoint correspondences"]

Keypoints, descriptors, and correspondences are distinct objects: points specify image coordinates, descriptors provide appearance evidence, and correspondences express the relationship between images. LoMa neither trains a new detector nor introduces image-pair interaction into independent feature extraction. This separation avoids recomputing database image descriptors for every query, although the matcher must still process each candidate image pair.

Key Designs¶

1. Multi-source geometric supervision: diversify data and rebuild inaccurate labels

Supervision comes from known relative poses and depth maps: these project keypoints from one image into the other to determine which detected locations correspond. The mixture does not merely add more MegaDepth photographs; it covers indoor, outdoor, aerial, object-centric, driving, forward-motion, and synthetic scenes. Table 1 lists 17 datasets, combining wide-baseline and optical-flow-style data with unequal sampling weights. Most major sources have weight 1 and several have 0.1; FlyingThings3D has 0.5, while SpatialVID, UnrealStereo4k, and Virtual KITTI 2 have 0.01. These are mixture weights, not dataset percentages, and they do not imply equal sample contributions from all sources. The goal is to expose features and matchers to more appearance changes and geometric relationships rather than the patterns of a single photograph collection.

The authors also perform additional geometric processing for three sources because matching supervision is sensitive to pixel-level label errors. MegaDepth (Re-MVS) reruns COLMAP MVS and includes subsequent sparse reconstruction models omitted by the original pipeline. MegaScenes is filtered to 303 scenes with sufficiently many cameras and 3D points before running COLMAP MVS. For SpatialVID, the existing 3D annotations are considered insufficiently accurate, so 59 scenes are selected for reconstruction. This branch runs SfM with SIFT+DaD keypoints, RoMa v2 correspondences, and shared intrinsics, then computes depth using RoMa v2 and a native PyTorch PatchMatch pipeline. Because forward motion dominates these scenes, initial pairs are not filtered on forward motion, avoiding frequent reconstruction failures. Data scaling therefore includes substantive label-generation and quality-selection costs, not just downloading more ready-made examples.

2. Descriptor pretraining: make independent features identify true matches in both directions

The descriptor follows DeDoDe, using 2048 DaD keypoints per image during training. Each descriptor depends only on its own image and does not yet know the other image, allowing offline computation and reuse. Pairwise descriptor dot products form a similarity matrix; a true correspondence should be distinctive in both matching directions rather than just one. Dual softmax serves this purpose by separately normalizing the two directions and multiplying their probabilities. To avoid corrupted glyphs in the extracted text, the following uses \(\alpha\) for inverse temperature and equivalently restates source Equations (2)-(3):

\[ P_{ij}=\operatorname{softmax}_{i}(\alpha S_{ij})\operatorname{softmax}_{j}(\alpha S_{ij}), \qquad \mathcal{L}_{\mathrm{desc}}=-\sum_{(i,j)\in\mathcal{M}_{A,B}}\log P_{ij}. \]

Here, \(S_{ij}\) is the descriptor dot product and \(\mathcal{M}_{A,B}\) is the set of true correspondences established by geometric supervision. The loss raises the probability of true matches in both directions, while nonmatching points compete through the softmax denominators. If a repetitive texture resembles many candidates, a high dot product alone is insufficient for an unambiguous bidirectional assignment. This objective follows DeDoDe; the paper's improvement primarily concerns data and training rather than a new contrastive loss. After this stage, the descriptor is frozen, so subsequent matcher training does not change the independent features.

3. Matcher scaling: give existing attention stronger inputs and more training data

The LightGlue-based matcher takes keypoint coordinates and descriptors, alternating within-image self-attention and cross-image cross-attention. Within-image interaction contextualizes a local feature using surrounding points; cross-image interaction constrains candidate associations with evidence from the other image. Self-attention uses RoPE so that attention scores depend on relative point positions; cross-attention does not use positional embeddings. Intuitively, relative layout is useful within an image, whereas pixel coordinates across viewpoints are not directly aligned. A learned linear projection connects descriptor and matcher embedding spaces when their dimensions differ.

The three main variants, B, L, and G, all use 9 Transformer blocks with attention-head dimension 64. Scaling changes embedding width to 256, 512, and 1024, respectively, rather than increasing depth. An additional B128 variant uses the lighter DeDoDe-B with 128-dimensional descriptors; the other main variants use DeDoDe-G descriptors. Each layer supervises the refined features for matching and also predicts whether each point has a true correspondence, termed matchability. A point without a counterpart should be recognized as unmatchable rather than forced onto a visually similar but incorrect candidate. Layer-wise supervision makes intermediate outputs meaningful, allowing fewer layers to be used at inference time for higher speed.

Default inference constructs the dual-softmax assignment from final-layer features and retains correspondences that are maxima in both their row and column. Matches with \(P_{ij}<0.1\) are then discarded; the result is sparse correspondences, not yet a fundamental matrix estimate or localization result. The paper also compares 3, 5, and 9 layers for speed and accuracy, but these are truncation settings, not a universal runtime for all scenarios. Rather than adding a complex new module, this design emphasizes how much capacity remains in an existing matcher when width, data, and supervision are sufficient.

A Worked Example¶

Consider a daytime and nighttime photograph of the same building; this example illustrates data flow and is not an additional experiment. Under the evaluation setting, each image first yields 4096 keypoints and independent descriptors, leaving the matcher to establish associations between two sets of 4096 candidates. Local brightness or window-frame texture alone may make a point resemble several candidates. Self-attention uses the surrounding keypoint layout, while cross-attention incorporates evidence from the other image to refine descriptors layer by layer. Ultimately, a window corner must be mutually preferred in both directions and have assignment probability at least 0.1 to become an output correspondence. Occluded points, points visible in only one image, and unresolved candidates should not be assumed to produce matches. For HardMatch, the predicted correspondences subsequently estimate a fundamental matrix, whose epipolar error is checked against manual correspondences. For localization, the correspondences enter the specified localization pipeline; the network does not directly produce the true camera pose in this example.

Loss & Training¶

Training first optimizes the descriptor, then fixes it while optimizing the matcher; it is not joint end-to-end training from detection through geometric estimation. Each matcher layer uses the same form of dual-softmax matching loss as the descriptor stage, plus binary cross-entropy for matchability. Ground-truth matchability indicates membership in the true correspondence set rather than whether the current model predicts a confident match. Both stages use AdamW, cosine annealing, \(560\times560\) resolution, and global batch size 64. The peak learning rate is \(2\times10^{-4}\), weight decay is \(5\times10^{-5}\), and EMA decay is 0.999. The descriptor trains for 50K steps, reportedly taking approximately one day on 8 A100 40GB GPUs. The matcher trains for 250K steps; B/L use 8 A100 40GB GPUs and G uses 16, each taking approximately two days. These times describe training resources, exclude the full data reconstruction cost, and are not runtime promises for consumer hardware. The authors report that longer descriptor training does not help and identify overfitting as a limitation, so more compute cannot be assumed to improve every stage.

Key Experimental Results¶

Main Results¶

The following selects results from source Table 2 on page 12; sparse methods use 4096 keypoints per image. MegaDepth-1500 and ScanNet-1500 report relative-pose error AUC@5°, while WxBS and HardMatch report mAA@10px; higher is better throughout. HardMatch estimates a fundamental matrix from predicted correspondences, computes PCK at different thresholds using epipolar errors on manual ground-truth points, and weights image pairs equally. mAA aggregates thresholded accuracy up to 10px; the main text does not specify the full threshold grid or robust-estimation parameters, referring to supplementary material not included in the available main-text source. HardMatch contains 1000 pairs from 100 categories, split by category into 10 validation and 90 test categories, with 8-28 manual correspondences per pair.

Method	MegaDepth AUC@5°	ScanNet AUC@5°	WxBS mAA@10px	HardMatch mAA@10px
MASt3R	42.4	33.6	34.5	33.6
RoMa	62.6	31.8	72.6	48.1
RoMa v2	62.8	33.6	64.8	46.5
ALIKED+LightGlue	48.1	14.5	43.9	35.7
LoMa descriptor, no learned matcher	51.7	18.7	63.0	39.5
LoMa-B	55.7	27.5	68.7	51.1
LoMa-L	56.5	29.3	70.6	53.5
LoMa-G	56.1	29.3	73.4	54.3

LoMa-G exceeds ALIKED+LightGlue by 18.6 points on HardMatch, but this does not imply that it surpasses dense methods on every geometric evaluation. For example, LoMa-L's MegaDepth AUC@5° of 56.5 remains below RoMa v2's 62.8; ScanNet likewise does not support a claim of universal superiority. Source Table 3 on page 13 reports 89.3 versus 67.9 for InLoc DUC2 at (1m, 10°), showing that improvements also transfer to localization. Those localization values depend on the HLoc pipeline and cannot be compared directly with HardMatch's pixel-threshold metric.

Ablation Study¶

The following reproduces source Table 5 on page 14, entirely using HardMatch validation mAA@10px rather than the test scores above. Changes accumulate from II onward; III expands only matcher training data, and IV subsequently expands descriptor training data.

Config	HardMatch validation mAA@10px	Change from preceding row
I: ALIKED+LightGlue	36.3	Baseline
II: DaD+DeDoDe+LightGlue, retrained on MegaDepth	42.0	+5.7
III: Matcher uses all data	48.9	+6.9
IV: Descriptor also uses all data	51.1	+2.2
V: Matcher training extended from 50K to 250K, LoMa-B	51.5	+0.4
VI: LoMa-L	52.8	+1.3
VII: LoMa-G	52.9	+0.1

Key Findings¶

Along this cumulative ablation path, expanding data raises the score from 42.0 to 51.1, a total of +9.1 points, exceeding the subsequent +0.4 from longer training alone.
Scaling from B to G adds +1.4 points overall, but L to G contributes only +0.1, showing that the marginal benefit of width is not constant.
Table 5 is a sequential ablation, not a full factorial experiment; these differences should not be treated as configuration-independent contributions.
The authors also report hundreds of pairs per second for LoMa-B on an A100 with 2048 points and batch size 16; this is matcher throughput, not full SfM throughput.

Highlights & Insights¶

The most valuable comparison retests an existing architecture after changing its training resources. It shows why performance under weak training conditions does not establish that local features have become obsolete.
Staged descriptor and matcher training preserves feature caching. For SfM or localization systems where each image participates in many pairs, this has engineering significance beyond single-pass accuracy.
HardMatch filters candidates using RoMa v2 confidence before manual matchability judgments and annotation. Difficulty is no longer defined entirely by previous reconstruction success, although selection remains method-dependent.

Limitations & Future Work¶

The authors acknowledge descriptor overfitting with longer training, and LoMa still struggles on HardMatch subgroups involving similar but distinct structures and extreme viewpoints.
HardMatch's fundamental-matrix evaluation requires static scenes and perspective cameras, while manual keypoints introduce error; it does not cover every nonrigid or nonperspective matching task.
Geographic and temporal biases remain, and filtering difficulty with RoMa v2 confidence does not guarantee neutrality across model families. The latter is a reader interpretation of the collection procedure.
The current method does not address in-plane rotation robustness; the main text mentions subsequent work, whose capabilities should not be attributed to these LoMa results.
The available material contains only the main paper; supplementary per-group tables, threshold grids, and additional detector ablations were not independently verified. The human baseline evaluates correspondences directly and is not directly comparable to models evaluated through a fundamental matrix.

Compared with DeDoDe and DaD: LoMa uses DaD keypoints and retrains DeDoDe descriptors without relearning the detector; its main evidence therefore concerns scaling descriptor and matcher training.
Compared with LightGlue: Backbone self-attention, cross-attention, and layer-wise supervision follow existing designs, while features, data, and width change. Accurate attribution matters more than presenting every component as new.
Compared with RoMa and RoMa v2: These methods perform dense matching, whereas LoMa retains sparse feature caching; HardMatch and relative-pose results show that their relative advantages depend on the task.
Research direction: Compare data sources, width, and training duration under fixed label quality and compute budgets, then examine transfer to rotation and dynamic scenes. This is a proposed follow-up, not an established result of the paper.

Rating¶

Novelty: 4/5. The contribution lies in scaling evidence, training data, and a challenging benchmark rather than a new network architecture.
Experimental Thoroughness: 5/5. The main paper covers matching, relative pose, localization, and cumulative ablations, with pipeline and training-resource differences still requiring care.
Writing Quality: 4/5. Claims connect clearly to evidence, although some reproduction details are deferred to supplementary material.
Value: 5/5. It provides a strong baseline for cacheable local feature systems and challenges the conflation of insufficient training with a paradigm ceiling.