Estimating Individual Tree Height and Species from UAV Imagery¶

Conference: ECCV 2026
Paper: ECCV Paper
Code: https://RolnickLab.github.io/DINOvTree
Area: Remote Sensing
Keywords: tree height estimation, tree species identification, drone imagery, forest monitoring, vision foundation model

TL;DR¶

This paper introduces BIRCH-Trees, the first benchmark for simultaneous tree height regression and fine-grained species classification from tree-centered monocular RGB UAV imagery, and proposes DINOvTree, an efficient multi-task framework with a shared foundation model backbone that outperforms separate task-specific models using nearly half the parameters.

Background & Motivation¶

Forests cover roughly 32% of global land area and serve as indispensable terrestrial carbon sinks and critical biodiversity reserves. Accurately quantifying carbon stocks in mature stands and tracking reforestation dynamics fundamentally hinges on tree-level biomass estimation, where species identity and diameter at breast height (DBH) represent the two primary allometric predictors. Traditional forest inventory relies heavily on manual field surveys, which are labor-intensive, logistically demanding, and unable to scale across vast forest tracts. While satellite remote sensing provides planetary-scale coverage, its spatial resolution remains far too coarse to resolve individual tree traits. Unoccupied Aerial Vehicles (UAVs) equipped with consumer-grade RGB cameras offer a scalable, cost-effective alternative for fine-grained canopy monitoring. However, direct measurement of tree height—a standard and reliable proxy for DBH—traditionally requires airborne LiDAR, whose prohibitive acquisition costs severely restrict operational deployment in resource-constrained tropical regions where biodiversity and carbon storage knowledge remain most incomplete.

Estimating single-tree structural and botanical traits from low-cost monocular RGB drone imagery introduces profound physical and computational bottlenecks. From a geometric standpoint, nadir UAV imagery captures only the upper envelope of the canopy with zero ground visibility, rendering monocular tree height estimation an ill-posed inverse problem; classical forestry approaches rely on species-specific allometric equations relating crown radius to height, yet these empirical power laws suffer from substantial local errors. Botanically, high intra-class morphological variability and high inter-class visual mimicry across diverse canopy architectures complicate fine-grained taxonomic differentiation, exacerbated by extreme class imbalance across natural forest communities. Prior deep learning studies have predominantly focused either on satellite-based dense canopy height models (CHMs) with notable sub-canopy errors, or on single-species UAV height estimation formulated as coarse classification bins, leaving a critical absence of standardized multi-biome benchmarks and unified multi-task architectures.

The entry point of this work is to isolate single-tree trait prediction from dense crown delineations, formulating simultaneous height regression and species classification under a unified framework across temperate, tropical, and boreal biomes. Core idea: establish the BIRCH-Trees benchmark and introduce DINOvTree, a multi-task architecture leveraging an end-to-end fine-tuned vision foundation model with decoupled cross-attention heads that independently steer spatial topology and fine-grained semantic representations, balanced dynamically via Dynamic Weight Average to achieve state-of-the-art height estimation and competitive taxonomy identification with nearly half the parameter footprint.

Method¶

Overall Architecture¶

DINOvTree takes as input a single RGB drone image tile \(I \in \mathbb{R}^{H \times W \times 3}\) centered on an individual pre-detected tree crown centroid. The primary objective is to simultaneously predict continuous tree height \(\hat{h} \in \mathbb{R}_{\ge 0}\) and class probabilities \(\hat{\mathbf{p}} \in [0, 1]^C\) across \(C\) candidate taxa using a single unified feed-forward pass. The pipeline is architecturally decoupled into three coordinated components: a shared vision foundation model (VFM) backbone, a spatially-guided height estimation head, and a multi-scale species classification head.

To capture versatile visual representations despite domain-specific data scarcity, DINOvTree adopts DINOv3 ViT pre-trained self-supervisedly on web-scale imagery as its shared feature extractor. Unlike linear probing configurations, the entire VFM backbone is fine-tuned end-to-end under joint task supervision to bridge the domain shift between generic web photographs and high-resolution aerial canopies. The extracted patch tokens are projected into task-specific subspaces, where learnable query tokens aggregate relevant spatial and semantic attributes via cross-attention, followed by a dynamic loss weighting strategy to ensure mutually beneficial multi-task convergence.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Tree-Centered RGB UAV Image"] --> B["Shared Vision Foundation Model Backbone<br/>End-to-End Fine-Tuned DINOv3"]
    B --> C["Patch Token Projection & Positional Encoding<br/>2-Layer MLP + 2D Sine Positional Encoding"]
    B --> D["Global Context Token<br/>Backbone [CLS] Extraction"]
    C --> E["Height Estimation Cross-Attention Head<br/>Learnable Height Query Aggregates Spatial Features"]
    C --> F["Species Classification Cross-Attention Head<br/>Learnable Class Query Aggregates Local Semantics"]
    D --> G["Multi-Scale Semantic Fusion<br/>Concatenation of Task Token and [CLS]"]
    F --> G
    E --> H["Linear Regression & Non-Negative Truncation<br/>Linear Layer + ReLU predicts continuous height ĥ"]
    G --> I["Linear Classification Projection<br/>Linear Layer + Softmax predicts species distribution p̂"]
    H --> J["Dynamic Weight Multi-Task Optimization<br/>Smooth L1 Loss + Cross-Entropy Loss + DWA"]
    I --> J

Key Designs¶

1. Shared foundation representation and spatially-aware projection: resolving ill-posed monocular height geometry Monocular height regression from top-down canopy perspectives requires delicate sensitivity to crown curvature, apical dominance, branch texture depth, and subtle shadow cues. Relying solely on global pooling or standard classification tokens obliterates localized geometric cues. DINOvTree employs a two-layer multi-layer perceptron (MLP with GELU non-linearities) to project the raw VFM patch tokens into an adapted latent space. For the height estimation head, 2D sine positional encodings are explicitly added to the projected key tokens. Because each image crop is strictly centered on the tree centroid, tree apexes and outer crown edges exhibit strong radial geometric consistency; explicit coordinate injection equips subsequent attention layers with relative spatial awareness from crown peak to perimeter, effectively alleviating geometric ambiguities caused by the absence of visible ground elevation.

2. Decoupled cross-attention queries with multi-scale semantic fusion: mitigating multi-task feature interference Hard parameter sharing in multi-task heads frequently triggers gradient conflict and negative transfer, degrading subtle fine-grained species boundaries. DINOvTree maintains strictly isolated task heads: in the height branch, a single learnable height query token cross-attends to the spatially encoded patch tokens, followed by a linear projection and a ReLU non-linearity ensuring non-negative scalar height predictions (\(\hat{h} \ge 0\)), avoiding the boundary saturation biases inherent in scaled Sigmoids. In the classification branch, a separate learnable classification query cross-attends to adapted patch tokens to distill localized leaf, needle, and branching textures. This distilled representation is then concatenated with the backbone's global [CLS] token prior to linear projection, synthesizing microscopic foliage patterns with macro-canopy architecture to maximize species separability.

3. Dynamic Weight Average multi-task optimization: balancing disparate regression and classification objectives Jointly optimizing continuous smooth L1 regression alongside discrete cross-entropy classification is notoriously susceptible to loss magnitude disparities and asynchronous convergence rates. DINOvTree avoids empirical hyperparameter grid searches by integrating Dynamic Weight Average (DWA). For the initial two training epochs, task weights are anchored at unity (\(\lambda_H = \lambda_S = 1\)). For subsequent epochs \(t \ge 3\), task weights are dynamically modulated based on the rate of loss reduction across successive epochs:

\[\lambda_k(t) = \frac{2 \exp(w_k(t-1) / T)}{\sum_{j \in \{H, S\}} \exp(w_j(t-1) / T)}, \quad \text{with } w_k(t-1) = \frac{\mathcal{L}_k(t-1)}{\mathcal{L}_k(t-2)}\]

With temperature set to \(T=2\), DWA adaptively scales up gradients for tasks exhibiting slower relative learning rates, preventing the shared ViT backbone from collapsing into a biased local optimum dominated by either regression or classification.

Loss & Training¶

The overall training objective is governed by the dynamic multi-task formulation:

\[\mathcal{L}(t) = \lambda_H(t) \mathcal{L}_H(\hat{h}, h) + \lambda_S(t) \mathcal{L}_S(\hat{\mathbf{p}}, c)\]

Height regression is supervised via the Smooth L1 loss:

\[\mathcal{L}_H(\hat{h}, h) = \begin{cases} 0.5(\hat{h} - h)^2, & \text{if } |\hat{h} - h| < 1 \\ |\hat{h} - h| - 0.5, & \text{otherwise} \end{cases}\]

Species classification is trained via standard categorical cross-entropy \(\mathcal{L}_S\). Optimization utilizes AdamW with decoupled weight decay, paired with a linear warmup and cosine annealing learning rate schedule, applying conservative learning rates to the pre-trained foundation backbone and higher learning rates to the task heads.

Key Experimental Results¶

Main Results¶

The BIRCH-Trees benchmark is evaluated across three ecologically distinct datasets—Quebec Trees (temperate forest), Barro Colorado Island (tropical forest), and Quebec Plantations (boreal plantation)—reporting mean and standard error across 5 random seeds:

Dataset	Method	Param (M)	Height \(\delta_{1.25}\) (%) ↑	Height MAE (m) ↓	Species F1 (%) ↑	Species Acc (%) ↑
Quebec Trees (Temperate)	Allometry (Tallo)	0	48.74	2.98	–	–
Quebec Trees (Temperate)	Mask R-CNN	88	\(74.34 \pm 0.32\)	\(1.73 \pm 0.02\)	\(65.49 \pm 0.45\)	\(65.86 \pm 0.60\)
Quebec Trees (Temperate)	ResNet-50	47	\(68.16 \pm 0.13\)	\(1.96 \pm 0.01\)	\(76.67 \pm 0.38\)	\(74.01 \pm 0.50\)
Quebec Trees (Temperate)	ConvNeXt-B	175	\(84.82 \pm 0.39\)	\(1.32 \pm 0.02\)	\(84.87 \pm 0.42\)	\(82.91 \pm 0.40\)
Quebec Trees (Temperate)	PECore-B	187	\(84.85 \pm 1.03\)	\(1.29 \pm 0.04\)	\(86.82 \pm 0.33\)	\(84.57 \pm 0.53\)
Quebec Trees (Temperate)	DINOv3-B (Separate)	171	\(86.35 \pm 0.26\)	\(1.25 \pm 0.01\)	\(\mathbf{87.46 \pm 0.53}\)	\(85.10 \pm 0.80\)
Quebec Trees (Temperate)	DINOvTree-B (Ours)	100	\(\mathbf{88.29 \pm 0.32}\)	\(\mathbf{1.17 \pm 0.02}\)	\(86.89 \pm 0.31\)	\(\mathbf{85.03 \pm 0.45}\)
Quebec Trees (Temperate)	DINOv3-L (Separate)	606	\(88.03 \pm 0.38\)	\(1.17 \pm 0.01\)	\(\mathbf{88.85 \pm 0.37}\)	\(86.92 \pm 0.57\)
Quebec Trees (Temperate)	DINOvTree-L (Ours)	328	\(\mathbf{89.28 \pm 0.28}\)	\(\mathbf{1.12 \pm 0.02}\)	\(88.38 \pm 0.38\)	\(\mathbf{87.25 \pm 0.36}\)
BCI (Tropical)	ConvNeXt-B	175	\(85.91 \pm 0.96\)	\(3.68 \pm 0.09\)	\(45.99 \pm 0.55\)	\(48.16 \pm 0.63\)
BCI (Tropical)	DINOv3-B (Separate)	171	\(85.74 \pm 0.69\)	\(3.61 \pm 0.07\)	\(46.44 \pm 1.32\)	\(46.44 \pm 1.32\)
BCI (Tropical)	DINOvTree-B (Ours)	100	\(\mathbf{91.25 \pm 0.40}\)	\(\mathbf{3.11 \pm 0.08}\)	\(\mathbf{48.96 \pm 0.85}\)	\(\mathbf{50.22 \pm 1.00}\)
BCI (Tropical)	DINOvTree-L (Ours)	328	\(\mathbf{94.14 \pm 0.23}\)	\(\mathbf{2.75 \pm 0.01}\)	\(\mathbf{54.39 \pm 0.67}\)	\(\mathbf{55.58 \pm 0.63}\)
Quebec Plantations (Boreal)	DINOv3-B (Separate)	171	\(\mathbf{86.71 \pm 0.39}\)	\(\mathbf{0.37 \pm 0.00}\)	\(82.56 \pm 1.86\)	\(84.00 \pm 1.76\)
Quebec Plantations (Boreal)	DINOvTree-B (Ours)	100	\(85.13 \pm 0.40\)	\(0.38 \pm 0.00\)	\(81.88 \pm 1.20\)	\(83.97 \pm 0.45\)
Quebec Plantations (Boreal)	DINOvTree-L (Ours)	328	\(85.09 \pm 0.26\)	\(0.37 \pm 0.00\)	\(\mathbf{87.80 \pm 1.19}\)	\(\mathbf{87.63 \pm 1.33}\)

Ablation Study¶

Ablation experiments conducted on the Quebec Trees dataset evaluate specific architectural components and multi-task loss weighting strategies:

Configuration / Variant	Height \(\delta_{1.25}\) (%) ↑	Height MSLE (\(10^{-2}\)) ↓	Species F1 (%) ↑	Note
DINOvTree-B (full model)	\(\mathbf{88.29 \pm 0.32}\)	\(\mathbf{1.85 \pm 0.05}\)	\(\mathbf{86.89 \pm 0.31}\)	Optimal performance across all metrics
w/o MLP projection	\(87.80 \pm 0.30\)	-	\(85.00 \pm 0.32\)	F1 drops sharply by 1.89%, task space alignment suffers
w/o positional encoding	\(87.83 \pm 0.30\)	-	\(86.33 \pm 0.29\)	Loss of radial spatial prior degrades both tasks
w/o cross-attention token (cls)	\(88.04 \pm 0.37\)	-	\(85.92 \pm 0.29\)	Relying solely on [CLS] misses local foliage cues
w/o global [CLS] (cls)	\(88.49 \pm 0.26\)	-	\(85.76 \pm 0.18\)	Omitting macro context harms classification by 1.13%
w/ global [CLS] (height)	\(87.72 \pm 0.26\)	-	\(86.32 \pm 0.71\)	Injecting semantic [CLS] into regression adds minor noise
w/ Sigmoid activation (height)	\(87.75 \pm 0.34\)	-	\(86.03 \pm 0.24\)	Scaled Sigmoid induces edge-case regression biases
Linear probe on DINOv3 [CLS]	\(85.26 \pm 0.11\)	-	\(83.88 \pm 0.57\)	Minimal head without cross-attention causes severe drops
Equal Weighting (EW)	\(87.53 \pm 0.47\)	\(2.00 \pm 0.10\)	\(85.95 \pm 0.56\)	Static weighting fails to accommodate gradient scale drift
Uncertainty Weighting (UW)	\(87.45 \pm 0.86\)	\(1.97 \pm 0.10\)	\(86.42 \pm 0.39\)	Unstable homoscedastic uncertainty optimization
DWA + PCGrad	\(88.09 \pm 0.14\)	\(1.91 \pm 0.02\)	\(86.59 \pm 0.53\)	Gradient projection yields no gain, adds compute overhead

Key Findings¶

Crucial role of full foundation model fine-tuning: Frozen backbones perform unacceptably poorly on drone imagery (frozen DINOv3-L achieves only 58.28% \(\delta_{1.25}\) and 28.76% F1). Strikingly, web-pretrained foundation models outperform satellite-pretrained counterparts (DINOv3-L-Sat achieves only 53.07% \(\delta_{1.25}\)), revealing that centimeter-level UAV crowns demand fine-grained natural object representations over low-resolution spectral patterns.
Superior parameter efficiency: Compared against maintaining two distinct, uncoupled models for height and species, DINOvTree achieves competitive or superior metrics across all benchmarks using only 54% to 58% of the total parameter count, demonstrating clear positive transfer on challenging tropical biomes where BCI height accuracy improves by over 5.5% percentage points.
Systematic height prediction biases: Mean signed difference (MSD = \(\hat{h} - h\)) reveals minor positive bias on temperate stands (+0.40 m), negative underestimation on dense tropical canopies (-0.91 m), and near-zero drift on young plantations (+0.06 m), strongly tracking regional train-test distribution shifts. Species-level regressions indicate that canopy dominants (e.g., Populus, Acer rubrum, \(R^2 \ge 0.88\)) are estimated with high precision, whereas understory species (e.g., Acer pensylvanicum, \(R^2 = 0.60\)) present higher regression dispersion.

Highlights & Insights¶

Standardized multi-biome benchmark with rigorous boundary buffering: Establishing 0.1 characteristic length inward boundary buffering and 99th percentile height filtering successfully prevents adjacent overlapping trees from corrupting ground-truth LiDAR canopy heights.
Dual-scale query decoupling for multi-task vision: Cross-attention query tokens decouple local texture extraction from global canopy shape, allowing regression and classification heads to specialize without mutually destructive feature representations.
Seamless dynamic loss modulation: Dynamic Weight Average eliminates cumbersome hyperparameter tuning between continuous L1 regression and discrete cross-entropy, enabling stable end-to-end convergence.

Limitations & Future Work¶

Long-tail species evaluation variance: Highly diverse tropical ecosystems exhibit severe class sparsity; rare species with very few test instances incur wide variance across evaluation seeds, suggesting a need for few-shot adaptation techniques.
Cross-sensor and seasonal generalization: Current evaluations rely on spatial train/val/test splits within identical flight campaigns; testing robustness against varying flight altitudes, lighting conditions, phenological variations (e.g., autumn leaf drop), and differing camera sensors warrants future study.
Absence of elevation priors or depth foundation models: The architecture currently relies purely on single RGB orthomosaics; incorporating photogrammetric DSM elevation data or leveraging pre-trained monocular depth foundation models represents a promising trajectory.

vs Satellite-Based Canopy Height Models (Lang et al. / Tolan et al. / DINOv3 Sat): Satellite approaches predict dense spatial CHMs at coarse resolutions where single tree boundaries cannot be resolved; DINOvTree focuses on tree-centered centimeter-scale UAV imagery, delivering precise individual-tree height and taxonomic identity.
vs Two-Stage Mask R-CNN Height Estimation (Fu et al. / Hao et al.): Earlier UAV pipelines discretize height values into arbitrary classification bins, losing continuous distance metric relations; DINOvTree frames height estimation as direct continuous regression, halving estimation error.
vs Classical Allometric Equations (Tallo Database): Even when endowed with oracle ground-truth species labels and exact tree segmentation masks, empirical allometric equations yield poor height threshold accuracy (\(\delta_{1.25} \approx 48\% - 58\%\)); DINOvTree decisively shows that end-to-end deep learning from raw RGB visual morphology radically outperforms manual allometry.

Rating¶

Novelty: ⭐⭐⭐⭐ [First dedicated multi-task VFM benchmark and framework for individual tree height and species estimation from monocular drone imagery]
Experimental Thoroughness: ⭐⭐⭐⭐⭐ [Extensive benchmarking across temperate, tropical, and boreal biomes covering diverse baseline architectures with rigorous 5-seed statistics and ablations]
Writing Quality: ⭐⭐⭐⭐⭐ [Clear mathematical formulations, thoughtful dataset curation protocols, and insightful ecological error analyses]
Value: ⭐⭐⭐⭐⭐ [Substantially decreases operational dependence on expensive airborne LiDAR surveys for forest biomass and carbon stock estimation]