AerialMetric: Benchmarking and Adapting UAV Monocular Metric Depth Estimation in the Real World¶

Conference: ECCV 2026
arXiv: 2606.29716
Code: https://kuieless.github.io/AerialMetric-ECCV2026-page/
Area: 3D Vision
Keywords: Monocular metric depth estimation, UAV aerial photography, Aerial benchmark dataset, LoRA domain adaptation

TL;DR¶

This paper constructs a large-scale UAV aerial monocular metric depth benchmark, AerialMetric, consisting of four complementary subsets (52K real + 16K synthetic image-depth pairs). It systematically reveals the severe domain gap of existing SOTA models under aerial perspectives where δ1 is close to 0, and improves the aerial δ1 to over 84% through parameter-efficient fine-tuning of MoGe2 with LoRA, with almost no loss in ground generalization capability.

Background & Motivation¶

Monocular metric depth estimation plays a core role in recovering absolute metric geometric depth in scenarios such as UAV logistics delivery, infrastructure inspection, and environmental monitoring. In recent years, depth networks represented by Depth Anything, Metric3Dv2, MoGe2, and UniDepthV2 have made remarkable progress on ground scenes, with some models even demonstrating a considerable degree of zero-shot generalization. However, when these models are directly deployed to UAV aerial perspectives, their performance drops precipitously—δ1 of ZoeDepth, DepthPro, Metric3Dv2, etc., on multiple aerial subsets is almost entirely 0, meaning that there is basically no valid correspondence between their absolute metric depth and the ground truth. The root of the problem lies in the severe shift in training data distribution: the domain gap between ground perspectives (horizontal or slightly downward, limited depth range, dominated by roads and indoor scenes) and aerial perspectives (wide range of pitch angle variations, bird's-eye geometric patterns, extremely wide depth range) is much larger than expected. The inherently ill-posed nature of monocular depth estimation is further amplified in aerial scenes—ground scenes feature rich perspective vanishing points and horizontal lines for models to calibrate scales, whereas in aerial top-down views, these geometric cues are systematically weakened or even disappear.

Behind this systematic failure is the lack of large-scale, diverse datasets with reliable metric depth annotations in real aerial scenes. Existing aerial data are either purely synthetic with a large sim-to-real gap, provide only sparse LiDAR projections instead of dense depth maps, or are limited to a scale of less than a thousand images, which is far from sufficient to support reliable aerial depth model training and evaluation. Meanwhile, existing depth evaluation schemes only report a dataset-level aggregated metric, failing completely to reveal in which dimension (such as aerial altitude, pitch angle, or field of view) the model degenerates—a diagnostic capability that is precisely what aerial deployment scenarios need the most. In short, aerial metric depth estimation is in a vacant state in terms of both data infrastructure and evaluation methodology.

The goal of this paper is to systematically fill this infrastructure gap. Core Idea: Construct a large-scale aerial metric depth benchmark, AerialMetric, containing four complementary subsets—covering four types of data sources: real photogrammetry aerial data, RTK orthogonal decoupled control acquisition, high-fidelity synthetic rendering, and internet aerial videos. It systematically quantifies the aerial domain gap, validates the effectiveness of this dataset on MoGe2 through parameter-efficient fine-tuning with LoRA, and establishes the evaluation benchmark and adaptation paradigm for aerial metric depth estimation.

Method¶

Overall Architecture¶

The core contribution of AerialMetric does not lie in proposing a new network architecture, but in systematically advancing aerial depth estimation from three aspects: data infrastructure, evaluation protocols, and adaptation strategies. The dataset itself consists of four functionally complementary subsets: the Oblique subset provides large-scale real-world photogrammetry data (integrating 6 public datasets with 47K high-quality image-depth pairs) as the primary data source for training; the Decoupled subset independently varies pitch angle, flight altitude, and field of view using orthogonal sampling via an RTK centimeter-level positioning system, specifically designed to decouply analyze the impact of flight parameters on depth; the Synthetic subset utilizes Unreal Engine + AirSim + Cesium, as well as Google Earth Studio, to generate 16K high-fidelity synthetic image-depth pairs, covering long-tail scenarios such as extreme perspectives and complex lighting; the Wild subset filters 100 high-quality sequences from over 600 social media UAV videos, obtaining pseudo-metric depth labels through a human-in-the-loop scale calibration pipeline. Based on this dataset, the paper performs parameter-efficient fine-tuning on MoGe2-ViT-L using LoRA to obtain MoGe2-Aerial, mixing Oblique, Synthetic, and ground domain data (Hypersim, MVS-Synth, TartanAir) at an 80:15:5 ratio for weighted sampling during training, which adapts to the aerial distribution while avoiding catastrophic forgetting of ground scenes.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Data Sources"] --> B["AerialMetric Four-Component Dataset"]

    B --> C["Oblique Subset<br/>6 Photogrammetry Datasets<br/>47K Real Image-Depth Pairs"]
    B --> D["Decoupled Subset<br/>RTK Orthogonal Acquisition<br/>4 Scenes × 16 Flight Trajectories"]
    B --> E["Synthetic Subset<br/>UE4 + GES Rendering<br/>16K Synthetic Image-Depth"]
    B --> F["Wild Subset<br/>Internet Videos<br/>Human-in-the-Loop Scale Calibration"]

    C --> G["MoGe2-ViT-L + LoRA Fine-tuning<br/>Oblique:Synth:Ground = 80:15:5"]
    E --> G
    G --> H["MoGe2-Aerial<br/>Aerial Adapted Model"]

    D --> I["Decoupled Parameter Robustness Analysis<br/>Pitch / Altitude / FOV"]
    F --> I
    H --> I
    I --> J["Comprehensive Evaluation Across Aerial-Ground Scenes"]

Key Designs¶

1. Four-component complementary dataset architecture: multi-dimensional coverage of aerial depth scenes

Aerial depth estimation lacks large-scale real data, yet no single data source can cover the diverse requirements of aerial tasks. The Oblique subset integrates six city-level photogrammetry datasets (UrbanBIS, GauU-SceneV2, UAVScenes, UrbanScene3D, OpenDroneMap, and ESRI), covering three scene categories (City, Rural, Natural) across 25 sub-areas, and provides approximately 47K dense metric depth annotation pairs. The Decoupled subset makes up for the confounding variables that Oblique cannot control—by varying pitch angles (−90°/ −75°/ −60°/ −45°), flight altitudes (80m/120m), and fields of view (63°/83°) independently through orthogonal sampling with an RTK centimeter-level positioning module across 4 typical scenes (building, lawn, farm, factory). Each viewpoint forms 16 strictly controlled configuration combinations, enabling precise subsequent diagnosis of model degradation with respect to specific parameters. The Synthetic subset covers scenes that are difficult to collect in the real world while providing noise-free, exact depth at the Z-buffer level. The Wild subset addresses the issue of laboratory data failing to represent real-world deployments—100 real UAV videos cover 50+ cities across 30 countries, with scene diversity and out-of-domain extent far exceeding any existing benchmarks. The four subsets respectively address four dimensions: real-world scalability, parameter controllability, distribution extensibility, and out-of-domain generalization.

2. High-fidelity depth ground-truth construction: integrating LiDAR denoising and photogrammetry rasterization

The depth ground truth acquisition methods differ across data sources and each has its engineering pitfalls. For UAVScenes and GauU-SceneV2 equipped with LiDAR, the core issue is the laser penetration artifact—laser beams from background points penetrate sparse foreground objects, producing incorrectly large depth values. The paper's solution is first to reconstruct continuous triangular meshes from RGB images via DJI Terra, and then compare the LiDAR projected depth with the mesh facets point-by-point, discarding pixels with deviations exceeding 1.0m as occlusion noise. For UrbanBIS, UrbanScene3D, ODM, and ESRI which only provide RGB, dense meshes are reconstructed at the highest native resolution, followed by pose-guided rasterization to generate pixel-aligned metric depth maps. A critical quality assurance is the strict geographic isolation split: during train-test splitting of the Oblique subset, a 50m buffer zone is mapped out to exclude 4.4K frames surrounding the test region, avoiding data leakage between adjacent aerial frames in the same large scene. Quantitative validation on the PolyTech scene shows that this pipeline achieves a ground sampling distance of 1.63cm and a mean square reprojection error of 0.993 pixels.

3. Orthogonal decoupled sampling protocol: independently isolating the impact of perspective parameters on depth estimation

Conventional depth evaluation reports aggregated metrics that hide crucial information about exactly which parameter causes model degradation. The design of the Decoupled subset is essentially a multi-factor nested orthogonal experiment: 4 levels of pitch angle, 2 levels of altitude, and 2 levels of field of view, totaling 16 configuration combinations covering 4 types of scenes. In practice, this is implemented by planning automated flight paths via DJI Pilot 2—when changing one parameter, the remaining parameters are strictly kept fixed through RTK stationing. Experimental results reveal a phenomenon that was not previously systematically quantified: the baseline models suffer from severe asymmetric performance degradation under nadir views (−90°) and extreme oblique views (−45°), and increasing the altitude from 80m to 120m causes a precipitous drop in δ1; although changing the field of view from 63° to 83° alleviates this, it is insufficient to bridge the gap. MoGe2-Aerial maintains δ1 > 80% under all 16 configurations, with the radar chart showing an almost isotropic contour, indicating that it has acquired robustness to aerial perspective parameters.

4. LoRA parameter-efficient fine-tuning strategy: finding a trade-off between aerial adaptation and ground generalization

Intuitively, full-parameter fine-tuning is optimal on aerial data (Full FT achieves δ1 = 95.6% on Oblique), but the cost is catastrophic forgetting of the ground domain (ground δ1 drops from 76.8% to 56.7%, and ETH3D drops from 88.8% to 61.5%). The strategy of freezing the ViT backbone and only fine-tuning the scale head is even worse—the ground FOV and perspective priors carried by the frozen backbone produce severely distorted relative point clouds for aerial images, forcing the isolated scale head to predict extremely fluctuating scale shifts, which leads to training divergence. LoRA finds the optimal balance between the two: with rank r=96, aerial δ1 ≈ 84% while ground δ1 loses less than 3 percentage points. A noteworthy engineering detail is that LoRA training strategically disables dense local losses (normal loss, mask loss, patch loss) because the pretrained model already possesses excellent high-frequency structural priors, and keeping local losses would instead force the limited-capacity LoRA modules to fit local noise in the MVS ground truth, resulting in blurry predictions. Concentrating the LoRA capacity completely on correcting the unique global geometric distortions and metric scale offsets of the aerial domain ensures parameter usage is highly cost-effective.

Loss & Training¶

LoRA modules are injected into all attention layers (qkv, proj) and feed-forward layers (fc1, fc2) of MoGe2-ViT-L, with rank r=96, alpha=192, and dropout 0.1. An AdamW optimizer is used to train for 1,800 steps with a batch size of 32. A dynamic token range strategy scales the longer side of images to a range of 1200-3600 tokens to adapt to UAV images of various resolutions. The encoder learning rate is 1e-6, the decoder is 5e-5, with a linear warmup for 500 steps followed by polynomial decay. The loss function only retains the global affine-invariant loss (weight 1.0) and the metric scale loss (weight 1.5), while disabling dense local losses. Training data are weighted sampled at 80% Oblique, 15% Synthetic, and 5% ground domain data (Hypersim, MVS-Synth, TartanAir).

Key Experimental Results¶

Main Results¶

The key comparisons without GT intrinsics (closer to real-world deployment scenarios) are organized below:

Dataset	Scene	Metric	MoGe2 Zero-shot	MoGe2-Aerial	Gain
Oblique	City	AbsRel ↓	48.4	10.3	-79%
Oblique	City	δ1 ↑	5.1%	89.3%	+84.2 pp
Oblique	Rural	δ1 ↑	0.3%	81.2%	+80.9 pp
Decoupled	Building	δ1 ↑	26.6%	87.6%	+61.0 pp
Decoupled	Lawn	δ1 ↑	19.5%	94.0%	+74.5 pp
Decoupled	Farm	δ1 ↑	0.0%	54.2%	+54.2 pp
Wild	0-400m	AbsRel ↓	55.26	21.34	-61%
Wild	0-400m	δ1 ↑	9.5%	53.7%	+44.2 pp

MoGe2-Aerial remains competitive compared to zero-shot MoGe2 on seven ground benchmarks (NYUv2, KITTI, ETH3D, iBims, DDAD, DIODE, HAMMER), even showing slight improvements on KITTI (δ1 +4pp) and DDAD (δ1 +5pp), validating that the LoRA strategy effectively avoids catastrophic forgetting.

Ablation Study¶

Config	Oblique δ1	Decoupled δ1	Ground δ1	Note
Zero-shot MoGe2	7.9%	20.0%	76.8%	Baseline, aerial basically fails
Full FT	95.6%	86.0%	56.7%	Aerial optimal but severe ground forgetting
Freeze ViT	87.7%	82.5%	35.8%	Frozen backbone leads to significant ground degradation
LoRA r=64	82.9%	84.3%	80.7%	Balanced but slightly weaker in the aerial domain
LoRA r=96	84.4%	83.9%	79.6%	Best compromise
LoRA r=128	82.4%	84.2%	69.4%	Over-parameterization leads to ground regression
Only Oblique	87.4%	77.5%	77.6%	Good in aerial but weak on Decoupled
+ Synthetic	82.3%	84.2%	79.7%	Decoupled significantly improved
+ Ground data	84.4%	83.9%	79.6%	Ground forgetting controlled to a minimum

Key Findings¶

The largest performance gap stems from data rather than model architecture: Zero-shot MoGe2 only achieves an aerial δ1 of 7.9%, which leaps to 84%+ after a single round of LoRA fine-tuning. This indicates that the data gap is the primary bottleneck for current aerial depth estimation, and design of new architectures cannot substitute for the lack of in-domain training data.
LoRA r=96 is the sweet spot for achieving dual-domain generalization across aerial and ground: Full parameter fine-tuning is optimal for the aerial domain but introduces unacceptable ground forgetting; r=128 over-parameterization also leads to ground regression; and r=64 is slightly weak in the aerial domain—r=96 is just sufficient for the LoRA rank to express the aerial geometric shift without overfitting.
The Farm scene is a recognized hard nut to crack: Due to repetitive vegetation textures and the absence of strong geometric anchors, MoGe2-Aerial only achieves a δ1 of 54.2% in this scene. Interestingly, UniDepthV2 performs the highest on Farm instead (57.9%), presumably because its pretraining corpus contains more agricultural scene distributions.
Altitude and pitch angle are two independent axes of aerial depth degradation: The baseline models present non-monotonic degradation across both the −90° to −45° pitch and 80m → 120m altitude dimensions, while the field of view (63° → 83°) has a relatively minor effect—these two factors do not cancel each other out and must be handled independently.
The decision to disable dense local losses in LoRA is quantitatively supported: Retaining local losses causes LoRA to fit geometric noise in the MVS ground truth, leading to blurry edges, which aligns with expectations—since the pretrained model has already learned high-frequency structures, the limited capacity of LoRA should focus on global geometric correction.

Highlights & Insights¶

The four-component complementary design serves as a paradigm for benchmark papers: Each subset has a clear "reason to exist," unlike many datasets that simply stack scale and diversity. In particular, the orthogonal experimental design of the Decoupled subset can precisely localize under which viewpoint parameters the model degenerates.
The mesh-comparison processing of LiDAR penetration artifacts is practical and reproducible: Reconstructing the mesh and comparing point-by-point with a 1.0m threshold is logically clear and produces quantifiable effects, making it highly referable for peers working with LiDAR depth ground truth.
Geographic isolation split is a critical yet easily overlooked quality assurance in many benchmark papers: The design of using a 50m buffer zone to exclude 4.4K frames surrounding the test region prevents data leakage from "having the same source in different frames."
The human-in-the-loop scale calibration pipeline of the Wild subset is structurally novel and practical: Progressing from VLM preliminary filtering of landmarks to manual verification and then multi-annotator consensus to obtain scale factors provides a feasible scheme for acquiring pseudo-metric ground truth from massive internet videos at low cost.
The marginal improvement of MoGe2-Aerial on some ground benchmarks is noteworthy: The improvements of δ1 +4pp on KITTI and δ1 +5pp on DDAD suggest that training data diversity and depth scale diversity in AerialMetric conversely enhance model performance on certain ground scenes.

Limitations & Future Work¶

Zero coverage of extreme weather: The Oblique and Decoupled subsets are mainly collected in clear weather, with rain, snow, heavy fog, and nighttime scenes basically missing, which the authors acknowledge in their Future Work.
The Wild subset is pseudo-metric rather than strict ground truth: Although the pipeline of COLMAP sparse poses + DA3 multi-view depth + manual scale calibration has a verified AbsRel of only 1.19%, errors in distant and low-texture areas may be amplified, so using it as a strict evaluation benchmark still warrants caution.
The fundamental problem of the Farm scene remains unsolved: Vegetation areas lack geometric texture and reliable depth anchors. Pure monocular methods may have reached their limit here, and incorporating multi-view or temporal information might be necessary in the future.
The adaptation strategy is only validated on two architectures: Although MoGe2 and UniDepthV2 represent different designs, the adaptation effects on DepthPro, Metric3Dv2, etc., still need to be verified.
Future Directions: Extend to multi-view metric depth estimation to achieve stronger geometric consistency; supplement data collection under extreme weather and nighttime conditions; explore more efficient adaptation methods like Adapters.

vs Depth Anything / Metric3Dv2 / MoGe2 / UniDepthV2: These are ground-level SOTA monocular depth models with strong zero-shot generalization capabilities, but they fail systematically in the aerial perspective (with δ1 approaching 0). AerialMetric proves that data replenishment, rather than architectural design, is an effective way to close this domain gap.
vs UseGeo / WildUAV / OccuFly / UAVid-3D: Existing aerial depth datasets are either limited in scale (hundreds to thousands of images) or only provide sparse LiDAR projection depths. AerialMetric significantly surpasses them in scale (68K pairs), dimensions (4 subsets), and geometric precision (dense metric depth + strict geographic isolation).
vs ClaraVid / Mid-Air / SynDrone: Purely synthetic aerial data is limited by the sim-to-real gap. AerialMetric uses real Oblique data as the primary training force with synthetic data only as a supplement, achieving better real-domain generalization.
vs TanDepth: TanDepth proposes an adaptation method based on aerial view but is limited to restricted scenes. The large-scale benchmark of AerialMetric provides a supporting platform for more comprehensive cross-scene comparisons.

Rating¶

Novelty: ⭐⭐⭐⭐ [The combination of the four-component complementary design + orthogonal decoupled analysis framework + human-in-the-loop scale calibration is a solid systematic contribution, but essentially belongs to data infrastructure rather than new models or new theories]
Experimental Thoroughness: ⭐⭐⭐⭐⭐ [Covers 3 aerial subsets + 7 ground benchmarks + fine-grained ablations (fine-tuning strategies / data ratios / cross-architecture transferability) + full-space exploration of decoupled parameter robustness, with a systematic and convincing experimental design]
Writing Quality: ⭐⭐⭐⭐ [Clear structure, rich figures (radar charts, heatmaps, 3D point cloud comparisons) and tables, decoding a sufficient amount of experimental details]
Value: ⭐⭐⭐⭐⭐ [Fills a critical gap due to the long-term lack of large-scale real benchmarks in the field of aerial metric depth estimation, and is expected to become a standard evaluation platform in this direction. The LoRA adaptation strategy also provides a directly reusable baseline]