MCVL: Multi-Space Cross-View Learning for Aerial-Ground Person Re-Identification¶
Conference: ECCV 2026
Paper: ECCV Official
Area: Remote Sensing
Keywords: Aerial-Ground Person Re-Identification, Cross-View Representation Learning, Deformable Homography Transformation, Multi-Space Alignment, View Decorrelation Contrastive Learning
TL;DR¶
Addressing the severe viewpoint discrepancy and spatial-global feature gaps between UAV aerial and ground perspectives in AG-ReID, MCVL introduces an intermediate geometric projection space via Deformable Homography Transformation (DHT) combined with Cross-View Learning (CVL), alongside a memory-bank-guided View Decorrelation Contrastive Loss (VDCL) for view bias suppression, achieving state-of-the-art view-invariant and identity-discriminative representations with zero inference overhead.
Background & Motivation¶
With the rapid integration of Unmanned Aerial Vehicles (UAVs) into modern intelligent surveillance systems, conventional Person Re-Identification (Re-ID) restricted to homogeneous camera networks—either ground-to-ground or aerial-to-aerial—falls short in meeting comprehensive surveillance demands across complex dynamic scenarios. Ground-based CCTV cameras provide persistent, continuous observations from fixed positions but suffer from limited fields of view and severe blind spots; conversely, UAV aerial cameras offer high mobility and wide spatial coverage but typically capture targets from high-altitude oblique angles with transient flight durations. Aerial-Ground Person Re-Identification (AG-ReID) has thus emerged as a critical paradigm aimed at matching identical human targets across these heterogeneous camera views.
Nonetheless, AG-ReID exhibits distinctive difficulties rooted in extreme viewpoint discrepancies across altitudes and perspectives. The steep pitch angles of aerial cameras dramatically alter human body proportions, relative spatial arrangements of parts, and local appearance textures compared to ground-level views, thereby inducing a substantial cross-view feature distribution gap. Existing AG-ReID approaches largely overlook this spatial disparity, attempting instead to directly establish global semantic correspondences through auxiliary components such as pedestrian attributes, decoupled view tokens, or view-aware prompts. Consequently, training suffers from the lack of explicit geometric constraints, while relying on extra decoder or prompt modules during inference incurs significant computational latency and memory consumption, hindering real-time practical deployment.
To resolve these limitations, this work posits that bridging the cross-view gap requires explicitly modeling intermediate spatial geometric mappings while systematically stripping view-specific biases from global representations. The core idea is to establish an intermediate cross-view projection space via Deformable Homography Transformation (DHT) that couples global planar mapping with local deformable offsets, utilize Cross-View Learning (CVL) to minimize spatial feature discrepancies, and employ a prototype-driven View Decorrelation Contrastive Loss (VDCL) to eliminate view bias, yielding unified view-invariant yet discriminative representations with zero extra inference cost.
Method¶
Overall Architecture¶
The MCVL pipeline consists of four major components: the feature extraction backbone, the Deformable Homography Transformation (DHT) module, the Cross-View Learning (CVL) strategy, and the View Decorrelation Contrastive Loss (VDCL). Given paired images from aerial and ground views, a pre-trained Vision Transformer (ViT) first extracts visual embeddings comprising a global class token \(F_i^v\) and patch tokens \(f_i^v\). The DHT module then projects aerial patch features onto the ground-view plane and vice versa via deformable grid sampling, synthesizing intermediate cross-view representations \(f_i^{ag}\) and \(f_i^{ga}\). Upon these projected features, the CVL strategy enforces spatial identity consistency and cross-view contrastive discrimination. Concurrently, VDCL maintains a dynamic memory bank of view-specific prototypes to actively push global features away from view-dependent biases while pulling together same-identity features. At test time, all projection and decorrelation modules are discarded, leaving only the standard ViT backbone for lightweight cosine similarity retrieval.
%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
IN["Cross-View Input Images<br/>Aerial View / Ground View"] --> FE["Feature Extractor ViT<br/>Class Token F / Patch Tokens f"]
FE --> DHT["Deformable Homography Transformation DHT<br/>Global Homography Matrix + Deformable Offset Field"]
DHT --> CVL["Cross-View Learning CVL<br/>Identity Consistency Loss + Cross-View Contrastive Loss"]
FE --> VDCL["View Decorrelation Contrastive Loss VDCL<br/>Memory Bank Maintenance + View Prototype Push"]
CVL & VDCL --> OPT["Multi-Task Joint Optimization<br/>Identity + Triplet + CVL + VDCL"]
OPT --> INF["Lightweight Inference Deployment<br/>DHT & Memory Bank Discarded, Pure ViT Backbone"]
Key Designs¶
1. Deformable Homography Transformation: bridging coarse and fine-grained cross-view geometry
Standard planar homography transformation assumes rigid planar mapping, failing to capture the complex 3D human body deformations and localized spatial distortions caused by severe camera tilt. DHT overcomes this by combining a global 3x3 homography matrix with a learnable local deformable offset field. Given aerial patch feature map \(f_i^a \in \mathbb{R}^{C \times H \times W}\), a lightweight regression branch predicts 8 parameters to form homography matrix \(H\) with \(h_{33} = 1\), mapping homogeneous coordinate \((u, v, 1)^\top\) to normalized ground coordinates \((u', v')\):
$$
\begin{bmatrix} x' \ y' \ w' \end{bmatrix} \simeq H \begin{bmatrix} u \ v \ 1 \end{bmatrix}, \quad u' = \frac{x'}{w'}, \ v' = \frac{y'}{w'}
$$
To compensate for localized non-rigid body deformations, DHT introduces two lightweight convolutional branches predicting \(P\) candidate displacement offsets \(\{\Delta x_i(u, v)\}_{i=1}^P\) and corresponding softmax-normalized attention weights \(\alpha_i(u, v)\). The refined sampling location \(\tilde{\mathbf{x}}(u, v) = (u', v')^\top + \sum_{i=1}^P \alpha_i(u, v) \Delta x_i(u, v)\) is used with differentiable bilinear grid sampling and a lightweight residual network to obtain the projected feature \(f_i^{ag}\). Symmetrically, ground-to-aerial projection uses the inverse matrix \(H^{-1}\) to yield \(f_i^{ga}\).
2. Cross-View Learning: enforcing spatial consistency and perspective invariance
Unconstrained deformable transformation may lead to degenerative spatial artifacts. To ensure geometric fidelity and bridge the spatial feature gap, CVL introduces two complementary objectives. The Cross-View Identity Consistency Loss (CICL) penalizes spatial divergence between original features and their projected counterparts using Smooth L1 loss:
$$
\mathcal{L}{\text{CICL}} = \mathcal{L}}}(f_i^a, f_i^{ag}) + \mathcal{L{\text{smoothL1}}(f_i^g, f_i^{ga})
$$
Simultaneously, the Cross-View Contrastive Loss (CVCL) imposes margin-based metric constraints directly on the cross-projected feature pairs \((f_i^{ag}, f_i^{ga})\), pulling same-identity projections together while repelling distinct identities beyond margin \(m\):
$$
\mathcal{L}|_2) \right)^2 \right]
$$
This dual optimization aligns the intermediate spatial representations, enforcing identity preservation across dramatic perspective shifts.}} = \frac{1}{N} \sum_{i=1}^N \left[ y_i |f_i^{ag} - f_i^{ga}|_2^2 + (1 - y_i) \left( \max(0, m - |f_i^{ag} - f_i^{ga
3. View Decorrelation Contrastive Loss: debiasing view-specific cues in global representations
In the global embedding space, view-dependent background textures, lighting, and aerial pitch angles frequently act as shortcut representations. VDCL decouples these view-specific biases from identity semantics. A fixed-capacity memory bank stores historical global embeddings updated with momentum coefficient \(\mu = 0.9\):
$$
\hat{F}i^v \leftarrow \mu \hat{F}_i^v + (1 - \mu) F_i^v
$$
For each mini-batch, view prototypes \(\mathcal{P}_a\) and \(\mathcal{P}_g\) are calculated by averaging memory features for each view. The view decorrelation loss \(\mathcal{L}_{\text{VD}}\) suppresses view-dependent characteristics by penalizing cosine similarity between individual features and their corresponding view prototypes:
$$
\mathcal{L}_v)
$$
Concurrently, an InfoNCE-based contrastive loss }} = - \frac{1}{B} \sum_{i=1}^B \cos(F_i^v, \mathcal{P\(\mathcal{L}_{\text{CL}}\) brings global features of the same identity across aerial and ground views closer together while pushing negative identity pairs apart. Together, \(\mathcal{L}_{\text{VDCL}} = \mathcal{L}_{\text{VD}} + \mathcal{L}_{\text{CL}}\) ensures that global embeddings encode purely identity-salient information.
Loss & Training¶
The overall training objective combines standard metric learning losses with the proposed multi-space cross-view objectives: $$ \mathcal{L}{\text{Overall}} = \mathcal{L}}} + \mathcal{L{\text{Triplet}} + \beta \mathcal{L}}} + (1 - \beta) \mathcal{L{\text{CVCL}} + \lambda \mathcal{L} $$ where }\(\mathcal{L}_{\text{Identity}}\) is cross-entropy loss, \(\mathcal{L}_{\text{Triplet}}\) is soft margin triplet loss, and hyperparameters \(\beta\) and \(\lambda\) balance spatial alignment and global decorrelation.
The model adopts an ImageNet-pretrained ViT-Base backbone with \(256 \times 128\) input resolution. Training is conducted on a single NVIDIA TITAN Xp GPU for 120 epochs using SGD with cosine learning rate decay from \(8 \times 10^{-3}\) to \(1.6 \times 10^{-6}\). Mini-batch size is set to 128 (32 identities, 4 images each). During testing, all DHT modules, projected representations, and memory banks are completely omitted, and only the class token from the ViT backbone is utilized for fast cosine distance ranking.
Key Experimental Results¶
Main Results¶
The authors evaluated MCVL across four representative AG-ReID benchmarks: CARGO (large-scale synthetic), LAGPeR, G2APS-ReID, and AG-ReIDv1.
Performance on the comprehensive CARGO dataset across four standard evaluation protocols is summarized below:
| Method | Type | ALL Rank-1 (%) | ALL mAP (%) | G↔G Rank-1 (%) | G↔G mAP (%) | A↔A Rank-1 (%) | A↔A mAP (%) | A↔G Rank-1 (%) | A↔G mAP (%) |
|---|---|---|---|---|---|---|---|---|---|
| SBS | General Re-ID | 50.32 | 43.09 | 72.31 | 62.99 | 67.50 | 49.73 | 31.25 | 29.00 |
| BoT | General Re-ID | 54.81 | 46.49 | 77.68 | 66.47 | 65.00 | 49.79 | 36.25 | 32.56 |
| AGW | General Re-ID | 60.26 | 53.44 | 81.25 | 71.66 | 67.50 | 56.48 | 43.57 | 40.90 |
| ViT (Baseline) | Transformer | 61.54 | 53.54 | 82.14 | 71.34 | 80.00 | 64.47 | 43.13 | 40.11 |
| VDT | AG-ReID Specific | 64.10 | 55.20 | 82.14 | 71.59 | 82.50 | 66.83 | 48.12 | 42.76 |
| VIF | AG-ReID Specific | 65.71 | 57.46 | 83.93 | 74.19 | 82.50 | 66.98 | 51.25 | 44.55 |
| SeCap | AG-ReID Specific | 68.59 | 60.19 | 86.61 | 75.42 | 80.00 | 68.08 | 58.13 | 52.35 |
| MCVL (Ours) | Multi-Space Alignment | 70.51 | 62.92 | 86.61 | 77.86 | 82.50 | 72.41 | 60.62 | 53.74 |
Under the challenging cross-view A↔G protocol on CARGO, MCVL surpasses the ViT baseline by +17.49% Rank-1 and outperforms previous state-of-the-art SeCap by +2.49% Rank-1 and +1.39% mAP.
In cross-dataset transfer evaluations, MCVL proves highly resilient against domain and viewpoint shifts:
| Transfer Setting | Method | A→G Rank-1 (%) | A→G mAP (%) | G→A Rank-1 (%) | G→A mAP (%) |
|---|---|---|---|---|---|
| CARGO → AG-ReIDv1 | ViT (Baseline) | 1.59 | 1.95 | 3.01 | 2.31 |
| CARGO → AG-ReIDv1 | VDT | 19.33 | 11.81 | 15.38 | 11.73 |
| CARGO → AG-ReIDv1 | SeCap | 23.06 | 14.16 | 17.36 | 16.05 |
| CARGO → AG-ReIDv1 | MCVL (Ours) | 24.93 | 15.97 | 20.79 | 19.02 |
| LAGPeR → AG-ReIDv1 | ViT (Baseline) | 33.15 | 22.70 | 28.90 | 20.32 |
| LAGPeR → AG-ReIDv1 | VDT | 34.74 | 23.42 | 29.83 | 21.53 |
| LAGPeR → AG-ReIDv1 | SeCap | 37.93 | 24.96 | 30.87 | 22.99 |
| LAGPeR → AG-ReIDv1 | MCVL (Ours) | 51.45 | 36.42 | 43.24 | 33.75 |
Ablation Study¶
Ablations on CARGO dissect the contribution of each component within CVL and VDCL:
| Config | \(\mathcal{L}_{\text{CICL}}\) | \(\mathcal{L}_{\text{CVCL}}\) | \(\mathcal{L}_{\text{VD}}\) | \(\mathcal{L}_{\text{CL}}\) | ALL Rank-1 (%) | ALL mAP (%) | A↔G Rank-1 (%) | A↔G mAP (%) |
|---|---|---|---|---|---|---|---|---|
| Baseline (BL) | - | - | - | - | 61.54 | 53.54 | 43.13 | 40.11 |
| + CICL | ✓ | - | - | - | 63.46 | 56.08 | 46.88 | 43.25 |
| + CVCL | - | ✓ | - | - | 64.10 | 55.13 | 48.12 | 42.38 |
| + CVL (both) | ✓ | ✓ | - | - | 65.06 | 56.37 | 50.00 | 43.99 |
| + VD | - | - | ✓ | - | 65.38 | 56.86 | 51.88 | 45.20 |
| + CL | - | - | - | ✓ | 65.06 | 58.11 | 52.50 | 47.60 |
| + VDCL (both) | - | - | ✓ | ✓ | 68.91 | 61.76 | 55.62 | 50.58 |
| Full MCVL | ✓ | ✓ | ✓ | ✓ | 70.51 | 62.92 | 60.62 | 53.74 |
Comparing standard Homography Transformation (HT) and Deformable Homography Transformation (DHT): - Baseline: ALL Rank-1 61.54%, A↔G Rank-1 43.13%; - +HT + CVL: ALL Rank-1 63.78%, A↔G Rank-1 47.50%; - +DHT + CVL: ALL Rank-1 65.06%, A↔G Rank-1 50.00% (an improvement of +2.50% Rank-1 over rigid HT).
Computational Complexity Analysis¶
MCVL operates with high efficiency: - FLOPs: MCVL requires 11.05 G, strictly matching vanilla ViT (11.05 G) and drastically lower than SeCap (20.90 G); - Inference Latency: 3.7 ms per image for MCVL vs. 7.5 ms for SeCap; - GPU Peak Memory: 0.91 GB/batch during training vs. 1.46 GB/batch for SeCap (a 37.7% reduction).
Highlights & Insights¶
- Multi-Space Collaborative Alignment: Viewpoint gaps stem from both low-level 2D geometric perspective distortions and high-level semantic biases. MCVL constructs a closed-loop system bridging spatial feature space via DHT-projected alignment and global feature space via prototype-guided decorrelation.
- Zero Extra Inference Cost: Unlike prior AG-ReID models that depend on auxiliary prompt decoders or attribute predictors during inference, all geometric transformation branches and memory banks in MCVL serve strictly as training scaffolding, preserving vanilla ViT inference speed.
- Cross-Domain Transfer Generalization: On the challenging LAGPeR to AG-ReIDv1 zero-shot transfer benchmark, MCVL achieves a dramatic +13.52% Rank-1 leap over SeCap, proving that the model captures generalized view-invariant human structural priors rather than memorizing camera-specific calibration artifacts.
Limitations & Future Work¶
- Severe Orthogonal Pitch Occlusions: In near-nadir top-down drone perspectives, human bodies undergo extreme self-occlusion where heads obscure the torso and limbs. 2D planar deformable projection cannot hallucinate missing anatomical details; integrating 3D parametric human models (e.g., SMPL) presents a promising avenue.
- Complex Urban Background Clutter: While VDCL suppresses view-wide prototype biases, intense background texture shifts in dynamic urban scenes warrant future exploration into fine-grained part parsing or anatomical keypoint alignment.
Related Work & Insights¶
AG-ReID diverges from conventional Re-ID (e.g., BoT, AGW, TransReID) primarily through steep altitude and perspective differentials. Prior approaches explored attribute recognition branches, view-decoupling tokens (VDT), or view-aware prompts (SeCap), yet often neglected continuous spatial geometric transformations. While homography transformations have been utilized in multi-view pedestrian detection (e.g., MVDet) for static ground plane projections, MCVL is the first to reformulate homography into a learnable, deformable cross-view attention mapping mechanism for person re-identification, providing valuable methodology for aerial robotics and intelligent transportation surveillance.
Rating¶
- Novelty: 4.5 / 5.0 (First application of deformable homography transformation to cross-view AG-ReID alongside memory prototype decorrelation)
- Experimental Thoroughness: 4.5 / 5.0 (Extensive evaluations across CARGO, LAGPeR, G2APS-ReID, and AG-ReIDv1, covering cross-dataset transfer and complexity profiling)
- Writing Quality: 4.5 / 5.0 (Rigorous mathematical formulation, clear motivation, and disciplined narrative)
- Value: 4.5 / 5.0 (Zero inference overhead makes it exceptionally viable for edge-device deployment)