QVAM: Query-guided View-aware Adaptive Modulation for Aerial-Ground Person Re-Identification¶

Conference: ECCV 2026
Paper: ECCV Original
Code: https://github.com/Sakuraandroxy/QVAM
Area: Remote Sensing
Keywords: Aerial-Ground Person Re-Identification / Continuous Viewpoint Modeling / Adaptive Feature Modulation / Cross-View Prototype Alignment

TL;DR¶

To tackle severe and continuous viewpoint gaps between UAVs and ground cameras, QVAM discards coarse binary view labels and rigid orthogonality constraints, distilling fine-grained viewpoint cues via learnable view queries to adaptively modulate [CLS] token representations, and incorporates dual-view momentum prototype alignment to achieve state-of-the-art cross-view matching.

Background & Motivation¶

Aerial–Ground Person Re-Identification (AGPReID) aims to retrieve matching pedestrian identities across unmanned aerial vehicles (UAVs) and ground surveillance camera networks, serving as an indispensable foundation for intelligent security surveillance, large-scale search and rescue, and suspect tracking. Unlike conventional view-homogeneous camera networks, AGPReID faces extreme viewpoint disparities between high-altitude overhead perspectives and horizontal eye-level camera views. This drastic discrepancy causes severe cross-view feature misalignment: an individual's appearance under steep top-down angles changes dramatically compared to frontal views, sharply lowering intra-identity feature similarity while inducing spurious visual overlap across different identities, which ultimately degrades the generalization performance of standard ReID models.

To bridge this aerial–ground viewpoint gap, existing approaches (such as VDT and SeCap) predominantly rely on coarse binary aerial/ground classification supervision and enforce feature disentanglement using rigid orthogonality constraints. However, in practical UAV flight operations, varying altitudes and depression angles introduce continuous, fine-grained viewpoint transitions. Treating aerial imagery as a homogeneous class with a simplistic binary label fails to capture this continuous viewpoint drift, resulting in intra-view feature fragmentation. Furthermore, rigid orthogonality constraints are overly restrictive: viewpoint-specific features are naturally entangled with discriminative identity characteristics (e.g., hairstyle, shoulder geometry, and posture visible from above). Forcing a hard orthogonal partition inadvertently discards valuable shared semantics and identity-discriminative cues.

To overcome these dual limitations, this paper proposes Query-guided View-aware Adaptive Modulation (QVAM). Instead of enforcing hard partition and rigid orthogonality, QVAM distills continuous viewpoint cues through implicit query interactions and executes dynamic channel modulation, suppressing viewpoint-induced biases while preserving essential identity information. Core idea: distill fine-grained continuous viewpoint cues from local image patches using learnable view queries to dynamically predict channel modulation masks that filter out view biases from the [CLS] representation, combined with dual-view prototype alignment to pull aerial and ground distributions together.

Method¶

Overall Architecture¶

QVAM is built upon a standard Vision Transformer (ViT) backbone. An input pedestrian image from either an aerial or ground camera is divided into non-overlapping spatial patches and fed into the ViT encoder, outputting a global [CLS] token and a set of patch tokens. The framework then coordinates three core components: the View-aware Decoder (VAD) employs learnable view queries to interact with local patch tokens, distilling continuous viewpoint cues into view-aware query embeddings; the Adaptive Feature Modulation (AFM) module fuses identity context with view query embeddings to predict dynamic channel masks that scale the [CLS] token into a view-invariant yet identity-discriminative representation; finally, the Cross-View Prototype Alignment (CVPA) loss maintains dual-view dynamic memory banks to enforce multi-level alignment between batch centroids and opposing view prototypes.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    IN["Input Aerial-Ground Pedestrian Image"] --> VIT["ViT Backbone<br/>Extracts [CLS] and Patch Tokens"]
    VIT -->|"Patch Tokens"| VAD["View-aware Decoder VAD<br/>Learnable Queries Distill Continuous View Cues"]
    VIT -->|"[CLS] Token"| AFM["Adaptive Feature Modulation AFM<br/>Identity Injection + View Refinement to Predict Masks"]
    VAD -->|"View-aware Query Embeddings Q_view"| AFM
    AFM -->|"View-invariant Feature f_inv"| OUT["Cross-View Prototype Alignment CVPA<br/>Dual-view Momentum Banks + Batch & Memory Alignment"]
    OUT --> LOSS["Joint Optimization Objectives<br/>CE + Triplet + CVPA + Diversity & Regularization Losses"]

Key Designs¶

1. View-aware Decoder: Distilling continuous viewpoint cues with learnable queries

Prior methods rely on coarse binary labels (0/1) to supervise view-specific representations, which fail to perceive continuous viewpoint variations caused by UAV altitude and depression-angle shifts. VAD adopts a standard Transformer decoder architecture initialized with \(L\) learnable view queries \(\mathbf{Q} \in \mathbb{R}^{B \times L \times C}\). The queries first pass through Multi-Head Self-Attention (MSA) to capture contextual inter-query dependencies. Subsequently, using the queries as \(Q\) and the patch tokens \(\mathcal{T}_{\text{patch}} \in \mathbb{R}^{B \times N \times C}\) extracted by the backbone as \(K\) and \(V\), Multi-Head Cross-Attention (MCA) aggregates fine-grained spatial cues, followed by an FFN projection to output view-aware query embeddings \(\mathbf{Q}_{\text{view}}\):

\[\mathbf{Q}_{\text{view}} = \text{FFN}(\text{MCA}(\text{MSA}(\mathbf{Q}), \mathcal{T}_{\text{patch}}, \mathcal{T}_{\text{patch}}))\]

To prevent multiple queries from suffering mode collapse onto identical image areas, a Query Diversity Loss \(\mathcal{L}_{\text{div}}\) is introduced. It enforces orthogonality between normalized attention vectors \(\mathbf{a}_{i,j}\) and \(\mathbf{a}_{i,k}\) across distinct queries over the patch dimension, encouraging different queries to attend to complementary body parts (e.g., head, shoulders, torso, legs) and capture multi-scale structural deformations under changing depression angles:

\[\mathcal{L}_{\text{div}} = \frac{1}{B \cdot L(L - 1)} \sum_{i=1}^B \sum_{j=1}^L \sum_{k=1, k \ne j}^L (\mathbf{a}_{i,j} \cdot \mathbf{a}_{i,k})^2\]

2. Adaptive Feature Modulation: Coarse-to-fine identity-preserving channel modulation

Enforcing strict orthogonal projection on representations often damages identity-discriminative cues that are inherently correlated with view perspective. The AFM module adopts a dynamic channel weighting mechanism via two stages. First, identity context injection: a shared learnable mask token \(\mathbf{m} \in \mathbb{R}^{1 \times C}\) is broadcast across the batch and combined with the global [CLS] token \(\mathbf{x}_{\text{cls}}\) through an FFN, yielding an identity-conditioned intermediate mask \(\mathbf{m}' = \mathbf{m} + \text{FFN}(\mathbf{x}_{\text{cls}})\). Second, view-adaptive refinement: treating \(\mathbf{m}'\) as query and the contextualized view queries \(\text{MSA}(\mathbf{Q}_{\text{view}})\) as keys and values in cross-attention, the output is projected and mapped via a Sigmoid activation into continuous scaling factors \(\mathbf{M} \in \mathbb{R}^{B \times C}\) in \((0, 1)\):

\[\mathbf{M} = \sigma(\text{FFN}(\text{MCA}(\mathbf{m}', \text{MSA}(\mathbf{Q}_{\text{view}}), \text{MSA}(\mathbf{Q}_{\text{view}}))))\]

Using the predicted mask \(\mathbf{M}\), the [CLS] representation is modulated via element-wise multiplication into a view-invariant feature \(\mathbf{f}_{\text{inv}} = \text{FFN}(\mathbf{x}_{\text{cls}} \odot \mathbf{M})\) and suppressed residuals \(\mathbf{f}_{\text{res}} = \text{FFN}(\mathbf{x}_{\text{cls}} \odot (1 - \mathbf{M}))\). To prevent mask weights from clustering around 0.5 where channels remain ambiguous, a Mask Entropy Regularization Loss \(\mathcal{L}_{\text{reg}}\) pushes scaling factors away from the uncertain midpoint via soft penalization:

\[\mathcal{L}_{\text{reg}} = -\frac{1}{B \cdot C} \sum_{i=1}^B \sum_{j=1}^C \left[ \mathbf{M}_{i,j} \log(\mathbf{M}_{i,j} + \epsilon) + (1 - \mathbf{M}_{i,j}) \log(1 - \mathbf{M}_{i,j} + \epsilon) \right]\]

3. Cross-View Prototype Alignment: Bridging modality distribution gaps via dual-view memory banks

In-network modulation alone cannot fully bridge statistical distribution shifts across heterogeneous modalities. CVPA constructs dual-view memory banks \(\mathbf{M}^A, \mathbf{M}^G \in \mathbb{R}^{I \times C}\) to store view-specific identity prototypes for aerial and ground views. During each training iteration, batch centroids \(\boldsymbol{\mu}_i^A\) and \(\boldsymbol{\mu}_i^G\) are calculated for each identity \(i\). Contrary to conventional contrastive memory setups that update prototypes post-loss to prevent leakage, QVAM updates prototypes prior to loss computation via momentum: \(\mathbf{p}_i^v \leftarrow \alpha \mathbf{p}_i^v + (1 - \alpha) \boldsymbol{\mu}_i^v\). Because alignment is strictly performed against prototypes from the opposing view, there is zero risk of same-view leakage, and this ordering significantly mitigates cold-start instability.

Alignment operates at two complementary levels: batch-level alignment \(\mathcal{L}_{\text{batch}}\) minimizes the Euclidean distance between aerial and ground centroids of identity \(i\) within the current batch; memory-level alignment \(\mathcal{L}_{\text{mem}}\) pulls batch centroids toward historical prototypes of the opposing view (\(\mathbf{p}_i^G \in \mathbf{M}^G\) for aerial, \(\mathbf{p}_i^A \in \mathbf{M}^A\) for ground):

\[\mathcal{L}_{\text{batch}} = \sum_{i \in \mathcal{I}_{\text{batch}}} \|\boldsymbol{\mu}_i^A - \boldsymbol{\mu}_i^G\|_2^2\]

\[\mathcal{L}_{\text{mem}} = \sum_{i \in \mathcal{I}_{\text{batch}}} \left( \|\boldsymbol{\mu}_i^A - \mathbf{p}_i^G\|_2^2 + \|\boldsymbol{\mu}_i^G - \mathbf{p}_i^A\|_2^2 \right)\]

The joint objective \(\mathcal{L}_{\text{CVPA}} = \mathcal{L}_{\text{batch}} + \mathcal{L}_{\text{mem}}\) ensures compact and aligned representation manifolds across views.

Loss & Training¶

The overall training objective combines backbone supervision, view-invariant representation constraints, prototype alignment, and regularization terms:

\[\mathcal{L}_{\text{total}} = \mathcal{L}_{\text{cls}} + \mathcal{L}_{\text{inv}} + \gamma \mathcal{L}_{\text{CVPA}} + \beta \mathcal{L}_{\text{div}} + \xi \mathcal{L}_{\text{reg}}\]

Specifically, \(\mathcal{L}_{\text{cls}}\) applies Cross-Entropy and Triplet losses to the backbone [CLS] token to preserve baseline discriminability; \(\mathcal{L}_{\text{inv}}\) applies Triplet loss to intermediate modulated features and both Cross-Entropy and Triplet losses to final view-invariant features \(\mathbf{f}_{\text{inv}}\). Hyperparameters are configured as momentum \(\alpha = 0.9\), \(\beta = 1\), \(\gamma = 1\), and \(\xi = 0.01\). The model is trained on a single NVIDIA GeForce RTX 3080 Ti GPU using an ImageNet-pretrained ViT-B/16 backbone. Images are resized to \(256 \times 128\). Training uses SGD with batch size 128 (32 identities, 4 images each), initial learning rate \(8 \times 10^{-3}\), and cosine annealing decay.

Key Experimental Results¶

Main Results¶

QVAM was comprehensively evaluated on three standard AGPReID benchmarks: CARGO, AG-ReID, and AG-ReIDv2. The main quantitative comparisons are summarized below:

Dataset / Protocol	Metric	Ours (QVAM)	Prev. SOTA (SeCap / VIF)	Gain
CARGO (ALL)	Rank-1 / mAP / mINP	78.85% / 71.02% / 60.25%	68.59% / 60.19% / - (SeCap)	+10.26% / +10.83%
CARGO (A↔G)	Rank-1 / mAP / mINP	72.50% / 64.41% / 52.78%	69.43% / 58.94% / - (SeCap)	+3.07% / +5.47%
CARGO (A↔A)	Rank-1 / mAP	85.00% / 79.63%	80.00% / 68.08% (SeCap)	+5.00% / +11.55%
CARGO (G↔G)	Rank-1 / mAP	91.07% / 82.62%	86.61% / 75.42% (SeCap)	+4.46% / +7.20%
AG-ReID (A→G)	Rank-1 / mAP / mINP	85.53% / 76.81% / 53.04%	84.03% / 76.16% / - (SeCap)	+1.50% / +0.65%
AG-ReID (G→A)	Rank-1 / mAP / mINP	88.46% / 80.52% / 55.12%	87.32% / 79.19% / 52.98% (VIF)	+1.14% / +1.33% / +2.14%
AG-ReIDv2 (A→C)	Rank-1 / mAP	88.41% / 82.29%	88.12% / 80.84% (SeCap)	+0.29% / +1.45%
AG-ReIDv2 (W→A)	Rank-1 / mAP	87.78% / 82.43%	87.56% / 80.15% (SeCap)	+0.22% / +2.28%

Ablation Study¶

Ablation experiments on CARGO systematically examine the contribution of each module across all four protocols:

Config	ALL (Rank-1 / mAP)	A↔G (Rank-1 / mAP)	A↔A (Rank-1 / mAP)	G↔G (Rank-1 / mAP)	Note
ViT Backbone	71.15% / 62.21%	59.38% / 53.46%	82.50% / 68.04%	83.93% / 75.54%	Baseline ViT-B/16
+ VAD	74.36% / 67.63%	64.38% / 59.86%	80.00% / 74.08%	86.61% / 79.74%	View-aware Decoder only
+ AFM	75.64% / 67.20%	68.12% / 60.99%	82.50% / 72.62%	84.82% / 76.76%	Mask modulation from [CLS] only
+ CVPA	74.68% / 70.00%	66.87% / 62.98%	82.50% / 78.59%	85.71% / 80.23%	Dual-view prototype alignment only
+ VAD + AFM	75.96% / 67.61%	68.12% / 60.34%	80.00% / 75.64%	85.71% / 78.05%	Query-guided feature modulation
Full Model (QVAM)	78.85% / 71.02%	72.50% / 64.41%	85.00% / 79.63%	91.07% / 82.62%	Full integrated framework

Key Findings¶

Crucial role of fine-grained viewpoint cues: When VAD is combined with AFM, A↔G Rank-1 improves by +3.74% over AFM-only (from 64.38% to 68.12%), demonstrating that learnable queries effectively capture subtle viewpoint deformations that global tokens cannot resolve.
Synergy between modulation and distribution alignment: Applying CVPA on top of VAD+AFM further elevates A↔G Rank-1 by +4.38% (from 68.12% to 72.50%) and boosts same-view metrics (A↔A Rank-1 +5.00%, G↔G Rank-1 +5.36%), validating that prototype alignment stabilizes batch sampling noise and anchors representations globally.
Hyperparameter sensitivity and viewpoint feature duality: Performance drops sharply when mask regularization \(\xi > 0.1\). This confirms the paper's insight that viewpoint-sensitive channels still carry identity cues; overly harsh binarization or suppression destroys useful information. Setting query count \(L = 64\) provides the optimal coverage; increasing to \(L = 128\) degrades accuracy due to redundant background attention.

Highlights & Insights¶

Rethinking view disentanglement: The paper challenges the common dogma of binary view labeling paired with rigid orthogonal feature separation in AGPReID. It demonstrates that continuous viewpoint drift cannot be modeled binarily and that soft, query-conditioned channel modulation provides a superior trade-off between bias removal and identity preservation.
Unsupervised semantic query localization: Without requiring bounding boxes or part annotations, the simple query diversity loss \(\mathcal{L}_{\text{div}}\) successfully guides learnable queries to distribute attention across distinct anatomical regions (head, shoulders, torso, legs), adapting to perspective changes automatically.
Leakage-free pre-loss memory momentum update: Leveraging the cross-modality property where queries only align against the opposing view's prototypes, updating memory banks prior to loss computation circumvents self-matching leakage while completely resolving cold-start volatility.

Limitations & Future Work¶

Increased training overhead: Introducing Transformer decoder layers and dynamic cross-attention computations increases memory consumption and training time compared to vanilla single-branch ViT architectures.
Adverse conditions and extreme occlusions: Experiments primarily assume clear daytime visibility. Under extreme conditions such as night-time infrared surveillance or severe canopy occlusions, local patch cues may degrade, affecting the quality of distilled viewpoint embeddings.
Extension to multi-modal aerial-ground retrieval: The query-guided adaptive modulation paradigm can naturally be generalized to text-to-aerial-video retrieval or cross-spectral RGB-IR AGPReID tasks.

vs VDT (CVPR 2024): VDT introduces a view token on [CLS] with orthogonal loss for binary view disentanglement. QVAM demonstrates that rigid orthogonality discards identity cues and replaces it with learnable queries that capture continuous viewpoint drift via adaptive soft modulation.
vs SeCap (CVPR 2025): SeCap designs self-calibrating prompts to discover view-invariant local features. QVAM focuses on adaptive modulation of global [CLS] representations paired with dual-view prototype alignment, outperforming SeCap by over 10% Rank-1/mAP on CARGO ALL.
vs VIF (ICCV 2025): VIF employs patch-level RotateMix data augmentation to simulate perspective changes. QVAM addresses the challenge from a representation learning and modulation architecture standpoint, making the two approaches complementary.

Rating¶

Novelty: ⭐⭐⭐⭐ [Insightful critique of binary labeling and rigid orthogonality, effectively replaced by query-guided continuous view modulation]
Experimental Thoroughness: ⭐⭐⭐⭐⭐ [Extensive evaluations across CARGO, AG-ReID, and AG-ReIDv2 with thorough ablations, sensitivity analyses, and attention visualizations]
Writing Quality: ⭐⭐⭐⭐⭐ [Clear motivation, coherent narrative, well-structured formulations and diagrams]
Value: ⭐⭐⭐⭐⭐ [Delivers double-digit improvements on complex aerial-ground benchmarks with open-source code]