Skip to content

content_hash: bd7ed7c4a9322cdb

Rectified Embedding Flow Learning for Aerial Multi-view Geo-localization

Conference: ECCV 2026
Paper: ECCV Official
Code: https://github.com/rhaohur/REFL
Area: Remote Sensing
Keywords: Aerial Geo-localization, Multi-view Geo-localization, Flow Matching, Conditional Distribution Transport, Cross-view Alignment

TL;DR

To tackle fine-grained cross-view distribution shifts that textual instructions fail to resolve in dense, homogeneous geographic scenes, this paper proposes Rectified Embedding Flow Learning (REFL), which formulates cross-domain alignment as a directed conditional distribution transport via continuous Ordinary Differential Equation (ODE) trajectories, achieving state-of-the-art average R@1 of 44.83% and R@10 of 65.85% on the Aerial MVGL benchmark.

Background & Motivation

Aerial geo-localization has emerged as a cornerstone for large-scale spatial retrieval, drone-based navigation, and emergency disaster rescue. Compared to ground-based cross-view matching, which is severely hindered by visual occlusions, complex obstacles, and limited horizontal fields of view, aerial perspectives provide unconstrained coverage over large unmapped geographic areas. However, previous investigations into aerial geo-localization have remained fragmented across isolated single-retrieval paradigms, including natural language-guided satellite localization, pure visual cross-view drone-to-satellite matching, and natural language-guided drone localization. In open-world deployments, unimodal retrieval systems face pronounced uncertainty: visual representations degrade under dynamic illumination and adverse weather, natural language instructions often introduce semantic ambiguity, and single-sensor failure paralyzes the entire system. Unifying heterogeneous viewpoints and modalities into a directed retrieval paradigm—termed Aerial Multi-view Geo-localization (AMGL)—is therefore imperative for robust spatial intelligence.

Nevertheless, applying universal multimodal retrieval paradigms (such as UniIR and MM-EMBED) to the AMGL setting encounters fundamental distributional barriers. Universal multimodal retrievers operate over open-domain datasets characterized by massive inter-category variance, where textual instructions supply coarse-grained global semantic shifts (adjusting macro mean \(\Delta\mu\) and variance \(\Delta\sigma^2\)) to separate distinct retrieval intentions (e.g., Image-to-Image vs. Image-to-Text). In stark contrast, AMGL data forms exceptionally dense, fine-grained sub-distributions within the geographic domain. Samples are densely clustered within visually and semantically homogeneous scenes consisting of shared environmental elements such as roads, trees, roofs, and building facades, drastically compressing the decision boundaries between positive and negative candidates. When a single query view (e.g., a low-altitude drone image) must simultaneously achieve view-level geometric alignment with satellite orthophotos and modality-level semantic alignment with text queries, static embeddings guided solely by textual prompts cannot model or reconstruct the complex non-linear cross-view distribution shifts.

This fundamental discrepancy indicates that cross-domain alignment in dense geographic embedding spaces requires continuous distribution transformations rather than static feature projections. The core idea is to formulate cross-domain multi-view alignment as a directed conditional distribution transport process, leveraging neural Continuous Normalizing Flows to first learn multi-view transformation priors to a shared latent manifold via velocity fields, and subsequently transport query embeddings along ODE trajectories to explicitly rectify fine-grained distribution shifts toward the target candidate domain.

Method

Overall Architecture

The REFL framework builds upon Qwen3-VL-Embedding-2B as the unified multimodal feature extractor across the joint domain set \(\mathcal{D} = \{U_i, U_t, S_i, S_t\}\) comprising drone/satellite imagery and text. The pipeline operates via a two-stage paradigm: Stage 1 introduces Velocity-Prior Flow Learning (FL), where the multimodal backbone remains frozen while prior encoders extract Gaussian distribution statistics to locate a shared latent manifold anchor \(\mathcal{M}_{z^*}\) via inverse variance weighting, supervising independent parameterized ODE velocity fields for each view domain. Stage 2 executes Trajectory-Guided Embedding Rectification (ER), which numerically integrates the query feature inversely along the source velocity field back to the shared latent manifold, followed by forward numerical integration along the target domain velocity field, producing a rectified query embedding that undergoes unidirectional contrastive learning with static candidate embeddings.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    In["Input: Query pair (xq, Pq) and Candidate target xc"] --> Enc["Unified MLLM Feature Extraction<br/>Extract initial embeddings fq and fc"]
    Enc --> S1["Velocity-Prior Flow Learning<br/>Anchor shared manifold M_z* via inverse variance weighting & fit view ODE fields"]
    S1 --> S2["Trajectory-Guided Embedding Rectification<br/>Reverse integration to manifold + Forward transport to target domain"]
    S2 --> Loss["Unidirectional Contrastive Learning<br/>Align rectified query embedding with static candidate embedding via InfoNCE"]

Key Designs

1. Velocity-Prior Flow Learning: Decoupling Multi-view ODE Fields via a Shared Latent Manifold

To bypass the prohibitive requirement of collecting strictly aligned multi-view triplets and text descriptions simultaneously, REFL adopts a joint manifold distribution assumption inspired by FlowBind. It posits that all individual view distributions originate from a latent shared manifold \(\mathcal{M}_{z^*}\). Given a matched feature pair \(f_q\) and \(f_c\) from view domains \(\mathcal{M}_q\) and \(\mathcal{M}_c\), a lightweight prior encoder estimates their respective multivariate Gaussian statistics \((\mu_m, \sigma_m^2)\). To integrate heterogeneous views according to their intrinsic uncertainty, Inverse Variance Weighting computes the parameters of the shared latent anchor \(z^*\): $\(\sigma_{z^*}^2 = \left(\sum_{m\in\{\mathcal{M}_q,\mathcal{M}_c\}}\frac{1}{\sigma_m^2}\right)^{-1}, \quad \mu_{z^*}=\sigma_{z^*}^2\sum_{m\in\{\mathcal{M}_q,\mathcal{M}_c\}}\frac{\mu_m}{\sigma_m^2}\)$ Reparameterization sampling yields the manifold anchor \(z^* = \mu_{z^*} + \epsilon \odot \sigma_{z^*}\) with \(\epsilon \sim \mathcal{N}(0, I)\). For each view domain \(m\), an independent MLP velocity field \(v_{\theta_m}\) fits the continuous ODE trajectory between \(z^*\) and \(f_m\). This formulation decouples pairwise cross-view interactions into standardized flows to and from a shared latent space, encoding continuous geometric and cross-modal transformation priors.

2. Trajectory-Guided Embedding Rectification: Two-Stage ODE Numerical Integration for Distribution Shift Compensation

To make continuous flow priors directly usable during feature retrieval without perturbing the static gallery, the ER mechanism treats cross-domain retrieval as a continuous conditional transport of the query embedding. During inference, gallery candidates remain static for fast vector index retrieval, while the query feature \(f_q \in \mathcal{M}_q\) undergoes two consecutive numerical ODE integrations. First, reverse integration from time \(t=1\) to \(t=0\) transports the feature back along the source view velocity field \(v_{\theta_{\mathcal{M}_q}}\) to strip away domain-specific biases and reach the shared latent state \(z_q^*\): $\(z_q^* = f_q + \int_{1}^{0} v_{\theta_{\mathcal{M}_q}}(z_t, t) dt\)$ Subsequently, conditioned on the retrieval intent specifying the target candidate domain \(\mathcal{M}_c\), forward integration from \(t=0\) to \(t=1\) drives the state along the target velocity field \(v_{\theta_{\mathcal{M}_c}}\): $\(\tilde{f}_q = z_q^* + \int_{0}^{1} v_{\theta_{\mathcal{M}_c}}(z_t, t) dt\)$ This explicit transport directly maps the query into the candidate distribution domain, compensating for severe perspective, altitude, and cross-modal discrepancies before vector matching.

3. Unidirectional Flow-Guided Contrastive Supervision: Static Candidate Alignment and Joint End-to-End Optimization

Rather than allowing both query and candidate feature distributions to dynamically shift—which would invalidate pre-computed candidate databases in practical retrieval engines—REFL establishes unidirectional InfoNCE contrastive learning (\(\mathcal{L}_{\text{ER-CL}}\)) exclusively between the rectified query \(\tilde{f}_q\) and static candidate representations \(f_c\). In Stage 2, joint optimization balances the velocity field regression loss \(\mathcal{L}_{\text{FL}}\) with \(\mathcal{L}_{\text{ER-CL}}\) using a balancing weight \(\lambda = 0.5\). LoRA fine-tuning on the MLLM backbone aligns the high-level semantic representation space with the underlying continuous ODE flow without inducing catastrophic forgetting of general multimodal knowledge.

Loss & Training

The framework adopts a decoupled two-stage training paradigm: In Stage 1 (Prior Flow Learning), the MLLM backbone is kept frozen while only the view-specific MLP velocity fields are trained. To prevent trivial solutions, the time horizon is decomposed into \(t \in (0, 1]\) and \(t = 0\): $\(\mathcal{L}_{\text{flow}}^{(m)} = \mathbb{E}_{t \sim \mathcal{U}(0,1]} \left\| v_{\theta_m}(z_{m,t}, t) - (f_m - \text{sg}(z^*)) \right\|_2^2\)$ $\(\mathcal{L}_{t_0}^{(m)} = \mathbb{E}_{t=0} \left\| v_{\theta_m}(z^*, 0) - (f_m - z^*) \right\|_2^2\)$ where \(z_{m,t} = t f_m + (1-t)\text{sg}(z^*)\), and \(\text{sg}(\cdot)\) denotes the stop-gradient operator. \(\mathcal{L}_{\text{flow}}^{(m)}\) stops gradients into \(z^*\) to isolate velocity field learning, whereas \(\mathcal{L}_{t_0}^{(m)}\) back-propagates into the prior encoder. The total flow loss is \(\mathcal{L}_{\text{FL}} = \sum_{m \in \{\mathcal{M}_q, \mathcal{M}_c\}} (\mathcal{L}_{\text{flow}}^{(m)} + \mathcal{L}_{t_0}^{(m)})\).

In Stage 2 (Trajectory-Guided Correction), LoRA adapters (\(r=64\), \(\alpha=128\)) are injected into the MLLM backbone, and the entire framework is jointly trained via: $\(\mathcal{L}_{\text{stage2}} = \lambda \mathcal{L}_{\text{FL}} + \mathcal{L}_{\text{ER-CL}}\)$ with \(\lambda = 0.5\), initial temperature \(\tau = 0.07\), and AdamW optimizer (\(lr = 3\times 10^{-5}\), weight decay \(0.01\)). ODE numerical integration uses 16 steps during both training and evaluation, providing fine transport precision with minimal latency.

Key Experimental Results

Main Results

Experiments were evaluated across all 10 sub-tasks on the Aerial MVGL Benchmark, encompassing natural language-guided drone localization, natural language-guided satellite localization, and visual cross-view matching. Baselines include domain-specific multi-view retriever GeoBridge, universal multimodal retrievers UniIR (with CLIP and XVLM backbones), VLM2Vecv2-2B, and Qwen3-VL-Embedding-2B.

Dataset / Task Direction Metric REFL (Ours) Qwen3-VL (Prev. SOTA) GeoBridge UniIR (XVLMfs)
GeoText-1652 (Task 1) \(U_t \to U_i\) R@1 / R@10 (%) 14.27 / 30.05 12.48 / 27.71 5.17 / 15.39 12.05 / 25.42
GeoText-1652 (Task 2) \(U_i \to U_t\) R@1 / R@10 (%) 23.85 / 64.68 18.31 / 58.57 0.96 / 8.60 15.92 / 51.88
VRSBench (Task 3) \(S_t \to S_i\) R@1 / R@10 (%) 15.26 / 48.06 12.31 / 40.72 2.79 / 16.13 11.89 / 43.82
FIT-RSFG (Task 3) \(S_t \to S_i\) R@1 / R@10 (%) 15.56 / 38.34 14.13 / 36.30 3.81 / 18.55 11.47 / 34.35
VRSBench (Task 4) \(S_i \to S_t\) R@1 / R@10 (%) 14.23 / 47.17 10.57 / 39.36 1.40 / 9.69 8.33 / 36.11
FIT-RSFG (Task 4) \(S_i \to S_t\) R@1 / R@10 (%) 18.79 / 42.34 16.56 / 40.87 3.57 / 18.46 13.18 / 38.06
University-1652 (Task 5) \(U_i \to S_i\) R@1 / R@10 (%) 78.64 / 96.70 70.71 / 94.98 13.20 / 43.90 59.99 / 89.48
SUES-200 (Task 5) \(U_i \to S_i\) R@1 / R@10 (%) 89.08 / 99.89 83.29 / 99.68 30.83 / 78.61 91.10 / 99.06
University-1652 (Task 6) \(S_i \to U_i\) R@1 / R@10 (%) 84.88 / 93.72 79.32 / 91.01 18.69 / 43.65 79.32 / 90.73
SUES-200 (Task 6) \(S_i \to U_i\) R@1 / R@10 (%) 93.75 / 97.50 90.00 / 93.75 40.00 / 66.25 92.50 / 95.00
Overall Average (Avg.) All 10 Tasks mR@1 / mR@10 44.83 / 65.85 40.77 / 62.29 12.04 / 31.92 39.57 / 60.39

Ablation Study

The module ablation investigates individual loss formulations, velocity pre-training, and embedding rectification combinations (Table 6), while ODE integration steps evaluate inference trade-offs (Table 9).

Config Static \(\mathcal{L}_{\text{CL}}\) Stage 1 FL Prior Rectified \(\mathcal{L}_{\text{ER-CL}}\) Stage 2 \(\mathcal{L}_{\text{FL}}\) mR@1 (%) mR@5 (%) mR@10 (%) Note
C0 - - - - 19.91 32.76 39.01 Zero-shot unadapted baseline
C1 - ✓ - - 16.15 26.15 32.44 Stage 1 prior only, without contrastive supervision
C2 ✓ - - - 40.77 55.34 62.29 Conventional static contrastive learning
C4 - - ✓ - 42.84 58.20 64.81 ER-CL without Stage 1 velocity pre-training
C5 - - ✓ ✓ 42.95 58.21 64.67 ER-CL + Stage 2 FL without Stage 1 pre-training
C6 - ✓ ✓ - 43.06 58.29 64.46 Stage 1 pre-trained, but no FL regularization in Stage 2
C7 (REFL Full) - ✓ ✓ ✓ 44.83 59.57 65.85 Complete two-stage flow learning and rectification

In the ODE step ablation (Table 9), setting the integration steps to 1, 2, 4, 8, 16, 32, and 64 yields mR@1 of 41.56%, 43.16%, 43.78%, 44.57%, 44.83%, 44.95%, and 44.95%, respectively. Accuracy rapidly plateaus at 16 steps, which is selected as the default configuration.

Key Findings

  • Embedding rectification breaks through static feature space bottlenecks: Full REFL (C7, 44.83% mR@1) outperforms conventional static contrastive learning (C2, 40.77% mR@1) by +4.06% mR@1 and +3.56% mR@10, confirming that non-linear distribution transport is critical for overcoming dense intra-scene aggregation.
  • Stage 1 velocity priors and Stage 2 regularization are complementary: Omitting Stage 1 pre-training (C5) causes a performance drop to 42.95% mR@1, whereas removing the flow regularization during Stage 2 (C6) degrades mR@1 to 43.06%. Both prior trajectory learning and joint contrastive regularization are necessary to keep transported representations physically calibrated.
  • Out-of-distribution robustness: When evaluated zero-shot across four unseen datasets (ERA, RSICD, RSITMD, and DenseUAV), REFL without the ER inference module achieves 19.74% average R@1, surpassing the Qwen3-VL baseline (17.46% R@1). This demonstrates that the two-stage training scheme enhances general representation learning without catastrophic forgetting.

Highlights & Insights

  • Formulating multi-view geo-localization as directed distribution transport: Instead of forcing multimodal embeddings into a static compromise space using text prompts, REFL uses continuous normalizing flows to model the dynamic transport field between view distributions.
  • Asymmetric unidirectional inference architecture: Applying numerical ODE integration strictly to the runtime query while maintaining static gallery representations preserves low-latency vector index retrieval while achieving non-linear cross-view correction.
  • Broad cross-task transferability: The concept of bridging disparate sensory domains via an anchor-based latent manifold and bi-directional ODE flow is directly transferable to other high-density fine-grained retrieval tasks, such as vehicle-to-infrastructure visual matching and infrared-to-visible cross-modal re-identification.

Limitations & Future Work

  • Vulnerability to extreme topological hard negatives: Visual inspections reveal that misretrievals predominantly arise in highly uniform urban grids or industrial parks where candidate sites share near-identical structural patterns and vegetation layouts, indicating that global flow transport still benefits from complementary local geometric keypoint constraints.
  • Fixed numerical ODE integration overhead: While 16 integration steps through lightweight MLPs add modest compute, high-throughput edge deployment would benefit from adaptive-step ODE solvers or flow distillation into one-step transport models.
  • Discrete viewpoint modeling: The current formulation categorizes camera perspectives into discrete UAV and satellite domains; integrating continuous physical parameters (such as flight altitude, camera pitch angles, and solar azimuth) directly into the velocity field network represents a promising frontier for continuous-view localization.
  • vs GeoBridge: GeoBridge requires strictly aligned ground-drone-satellite image triplets and text captions during training, resulting in severe data scarcity and rigid architectures. In contrast, REFL operates over flexible pairwise directed retrieval paths and outperforms GeoBridge substantially (44.83% vs. 12.04% average R@1).
  • vs UniIR (CLIPfs / XVLMfs) & VLM2Vec: Universal instruction-tuned retrievers rely on coarse semantic biases that fail to separate dense, fine-grained geographic distributions. REFL couples the semantic richness of MLLMs with continuous neural ODE velocity fields to explicitly transport query embeddings, delivering superior cross-view alignment.

Rating

  • Novelty: ⭐⭐⭐⭐⭐ Formulates multi-view geo-localization as continuous conditional distribution transport, breaking through the limits of static embedding spaces.
  • Experimental Thoroughness: ⭐⭐⭐⭐⭐ Comprehensive evaluations across the 10-task Aerial MVGL benchmark, coupled with ablation studies, zero-shot transfer tests, and qualitative visualizations.
  • Writing Quality: ⭐⭐⭐⭐⭐ Rigorous mathematical formulation, clear problem motivation, and coherent progression from motivation to methodology.
  • Value: ⭐⭐⭐⭐⭐ Establishes a foundational paradigm for low-altitude aerial perception, drone spatial retrieval, and cross-domain multimodal alignment.