Skip to content

Counting Trees from Satellite Imagery with Noisy Supervision

Conference: ECCV 2026
Paper: ECCV 2026 Poster
Code: https://github.com/dgominski/treematch
Area: Autonomous Driving
Keywords: tree counting, optimal transport, noisy supervision, satellite imagery, density estimation

TL;DR

Addressing ambiguous crown boundaries in dense canopies and noisy airborne LiDAR annotations, TreeMatch models satellite tree counting as Unbalanced Optimal Transport (UOT) density matching and bootstraps transport marginal residuals to iteratively self-correct noisy labels, outperforming detection and regression baselines on the multi-continent TinyTrees benchmark.

Background & Motivation

Estimating the number and spatial distribution of individual trees is foundational for environmental monitoring, reforestation tracking, biomass accounting, and vegetative infrastructure management. Most satellite-scale forestry observations have historically relied on coarse aggregate proxies such as forest cover percentages, mean canopy height, or total biomass, leaving tree-level structural analysis largely unaddressed. While individual trees can often be resolved in ultra-high-resolution aerial or UAV surveys, extending single-tree inventorying to satellite imagery faces acute physical and visual bottlenecks. At satellite ground sampling distances ranging between 0.8 m and 4.2 m, isolated trees remain distinguishable, but closed-canopy forests merge into continuous green textures with pervasive sensor blur and shadow interference. Under these conditions, the discrete boundaries of individual crowns become ill-defined, causing classical instance detection frameworks (e.g., YOLO or CenterNet) to collapse.

The practical challenge is compounded by severe annotation bottlenecks. Precise manual point or polygon annotations cannot scale globally due to prohibitive labor costs. Conversely, automated weak supervision derived from Airborne Laser Scanning (ALS) or canopy height models can scale to millions of hectares, but introduces structured, correlated noiseโ€”including widespread false positives in understory foliage and false negatives across dense canopies. Standard learning objectives fail under this noise regime: pixel-wise \(L_2\) regression or Kullback-Leibler divergences incur disproportionately large penalties for slight localization shifts and directly overfit label noise, whereas classical balanced Optimal Transport (OT) rigidly enforces mass conservation, creating catastrophic distortions when target labels under- or over-count tree instances.

The core insight is that satellite tree counting must decouple geometric proximity alignment from rigid mass conservation, permitting continuous spatial measure matching with built-in tolerance for count discrepancies. Core idea: formulate tree counting from satellite imagery as an Unbalanced Optimal Transport (UOT) spatial density matching problem, leveraging the marginal transport residuals to dynamically self-correct weak annotation noise during training while calibrating total scale with sparse manual labels.

Method

Overall Architecture

The TreeMatch framework adopts a UNet architecture with a ResNet-50 encoder that maps multi-spectral satellite image patches \(x \in \mathbb{R}^{H \times W \times C}\) to a continuous non-negative tree density map \(z \in \mathbb{R}^{H \times W}_+\). The total tree count across an image patch is directly retrieved via pixel summation. Training is orchestrated across two distinct data sources: a small, accurate strong set with manual point annotations and total counts, and an extensive weak set derived from automated ALS pipelines. Strong annotations supervise global count consistency alongside UOT geometric matching. For weak annotations, the framework first computes a non-differentiable UOT transport plan, extracts spatial marginal residuals to correct label omissions and spurious artifacts, and then optimizes the network via a second UOT loss evaluated between the prediction and the self-corrected target.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Satellite Multi-Spectral Image x"] --> B["UNet Density Network ฯ•"]
    B --> C["Predicted Continuous Density z"]
    C --> D["Unbalanced Optimal Transport<br/>Spatial Measure Alignment"]
    E["Weak Supervision y (ALS / Automated)"] --> F["Marginal Residual Computation<br/>r = ฮณโŠค1 - y (no grad)"]
    C -.-> F
    F --> G["Bootstrapped Target Correction<br/>yฬƒ(t) = y + ฮป(t)r"]
    G --> H["Weak UOT Loss Evaluation<br/>Back-propagated Gradient Update"]
    C --> H
    I["Strong Supervision y (Manual Points)"] --> J["Joint Strong Supervision Loss<br/>Global L1 Count + Geometric UOT"]
    C --> J

Key Designs

1. Unbalanced Optimal Transport Density Matching: Decoupling Geometry from Rigid Mass Conservation
In satellite imagery of forest canopies, precise tree centers exhibit unavoidable localization uncertainty. Standard pixel-wise metrics (such as MSE or cross-entropy) heavily penalize minor spatial displacements. While standard Optimal Transport (OT) addresses geometry through an explicit ground cost matrix (using squared Euclidean distance between 2D pixel coordinates \(C_{ij} = \|\text{pos}(i) - \text{pos}(j)\|_2^2\)), it enforces strict mass conservation by normalizing predictions and targets into probability distributions. In noisy satellite counting, forced mass equality compels the model to stretch transport mass across arbitrary distances to compensate for missing or redundant labels. TreeMatch adopts Unbalanced Optimal Transport (UOT), relaxing strict marginal constraints through generalized KL divergence penalties: $\(W_{C,\varepsilon,\tau}^{\text{UOT}}(z, y) = \min_{\gamma \ge 0} \langle C, \gamma \rangle + \varepsilon \mathrm{H}(\gamma) + \tau \Big( \mathrm{KL}(\gamma \mathbf{1} \parallel z) + \mathrm{KL}(\gamma^\top \mathbf{1} \parallel y) \Big)\)$ where \(\varepsilon > 0\) denotes entropic regularization (set to 0.005, corresponding to an interaction radius of \(\approx 0.07\) in normalized image coordinates) and \(\tau > 0\) controls the penalty for mass creation or destruction (set between 0.1 and 0.5). By penalizing mass discrepancies locally rather than banning them, UOT naturally absorbs counting uncertainty and label incompleteness while preserving spatial alignment.

2. Bootstrapping Marginal Residuals: Iterative Label Self-Correction from Transport Marginals
Automated ALS annotations contain localized errorsโ€”missing trees in tight clusters and false peaks over ground clutterโ€”that cannot be rectified by global thresholding. The optimal transport plan \(\gamma\) obtained from solving the UOT objective captures spatial discrepancy between predicted features and noisy annotations. Specifically, the column marginal \(\gamma^\top \mathbf{1}\) denotes the effective mass assigned to each annotation location under the geometric transport model. The marginal residual is defined as: $\(r = \gamma^\top \mathbf{1} - y\)$ Positive residual values (\(r_i > 0\)) reveal locations where the network predicts strong tree evidence that receives transport mass beyond original annotations (identifying false negatives), whereas negative values (\(r_i < 0\)) indicate annotated mass unsupported by image evidence (identifying false positives). The training target is dynamically updated via: $\(\tilde{y}(t) = y + \lambda(t) r\)$ where \(\lambda(t)\) smoothly ramps from 0 to \(\alpha = 0.8\) over the initial 400 optimization steps via a tempered sigmoid schedule. Crucially, the transport plan \(\gamma\) is solved with stop_grad applied, ensuring that the residual calculation operates strictly as a pseudo-label refinement step without introducing unstable recurrent gradient feedback.

3. Stratified Dual-Source Supervision: Grounding Global Scale with Diverse Representation
Relying entirely on weak labels risks global scale drift, whereas training exclusively on sparse manual labels leads to catastrophic overfitting on regional biomes. The training pipeline pairs a compact manual set \(\mathcal{D}^\text{strong}\) with a large-scale weak set \(\mathcal{D}^\text{weak}\) sampled in a stratified 1:1 ratio per batch. For strong samples, the network is constrained by a global count loss \(\mathcal{L}^\text{count} = |\sum_i \phi(x)_i - \sum_i y_i|\) and exact geometric UOT matching. For weak samples, only the self-corrected UOT loss is evaluated. This decoupled loss formulation anchors absolute count calibration to gold-standard labels while leveraging the massive spatial diversity of weak supervision.

Loss & Training

The total objective optimized across training batches is: $\(\mathcal{L} = \sum_{(x,y) \in \mathcal{D}^\text{strong}} \Big( |\sum_{i} \phi(x)_i - \sum_{i} y_i| + \mathcal{L}^\text{UOT}(\phi(x), y) \Big) + \sum_{(x,y) \in \mathcal{D}^\text{weak}} \mathcal{L}^\text{UOT}(\phi(x), \text{stop\_grad}(\tilde{y}))\)$ Optimization uses AdamW with an initial learning rate of \(8 \times 10^{-5}\) under a cosine annealing schedule for 500 epochs. Training on an entire regional split executes in under one hour on a single NVIDIA RTX 3090 GPU. At inference time, TreeMatch processes 99 million pixels per second, translating to \(887\ \text{km}^2/\text{s}\) throughput on PlanetScope imagery in a single forward pass.

Key Experimental Results

Main Results

The TinyTrees benchmark spans three continents, three satellite sensors, and \(25,890\ \text{km}^2\) of forest cover with over 216 million tree annotations (including 639k manual point verifications): China (Gaofen-2, 0.8 m GSD), Rwanda (PlanetScope, 3.4-4.2 m GSD), and France (SPOT-6, 1.5 m GSD). Performance evaluated on held-out test splits across standard counting metrics (RMSE in trees/hectare, \(R^2\), and nMAE) is summarized below (Table 3 from the paper):

Category Method Strong Weak China (Gaofen-2) RMSEโ†“ / \(R^2\)โ†‘ / nMAEโ†“ Rwanda (PlanetScope) RMSEโ†“ / \(R^2\)โ†‘ / nMAEโ†“ France (SPOT-6) RMSEโ†“ / \(R^2\)โ†‘ / nMAEโ†“
Regression Density regression (\(L_2\)) โœ“ 64.6 / 0.55 / 37.4% 79.3 / 0.36 / 54.0% 163.4 / 0.20 / 40.2%
Regression Count regression (Sum Pool) โœ“ โœ“ 67.9 / 0.50 / 43.7% 75.8 / 0.42 / 53.8% 149.2 / 0.33 / 39.1%
Regression Uncertainty-aware reg. โœ“ 63.3 / 0.57 / 37.9% 80.5 / 0.34 / 55.3% 163.0 / 0.20 / 40.6%
Detection YOLOv8 โœ“ - (diverged) / - / - 448.7 / -0.50 / 85.0% n/a (incompatible format)
Detection CenterNet โœ“ โœ“ 82.9 / 0.26 / 51.8% 98.9 / 0.01 / 69.8% 181.5 / 0.01 / 45.6%
Detection P2P โœ“ โœ“ 98.7 / -0.05 / 60.4% - (diverged) / - / - 177.2 / 0.06 / 45.6%
Detection TreeFormer โœ“ 68.7 / 0.49 / 41.2% 86.0 / 0.25 / 59.8% 179.7 / 0.03 / 45.1%
Matching Bayes. Crowd Count. โœ“ 168.8 / -2.06 / 134.1% 80.5 / 0.34 / 60.9% 162.0 / 0.21 / 43.7%
Matching VRSNet โœ“ 67.5 / 0.51 / 41.2% 81.7 / 0.32 / 56.9% 165.1 / 0.18 / 40.3%
Matching DM-Count (Balanced OT) โœ“ โœ“ 65.0 / 0.55 / 39.6% 76.8 / 0.40 / 53.4% 158.0 / 0.25 / 40.9%
Ours TreeMatch โœ“ โœ“ 60.6 / 0.60 / 36.6% 72.4 / 0.47 / 51.1% 147.2 / 0.35 / 37.4%

Ablation Study

Ablation experiments conducted on the challenging Rwanda tropical forest split evaluate supervision regimes, loss choices, and encoder architectures (Table 5 from the paper):

Ablation Focus Model Configuration RMSE (trees/ha) โ†“ \(R^2\) โ†‘ nMAE (%) โ†“ Note
Full Model TreeMatch (Full) 72.4 0.47 51.1% ResNet-50 + UOT + Marginal Residuals
(a) Supervision No Train-weak 76.1 0.41 52.9% Removing weak supervision degrades generalization
(a) Supervision No Train-strong 82.4 0.31 57.0% Removing strong supervision destroys scale calibration (+10.0 RMSE)
(a) Supervision Uniform sampling 71.4 0.48 50.8% Minor variance on Rwanda, less stable across sensors
(b) Loss Formulation \(\mathcal{L}^\text{count}\) only 75.5 0.42 53.5% Lacks fine-grained spatial density guidance
(b) Loss Formulation Balanced OT 78.3 0.38 53.6% Mass conservation penalizes counting noise excessively
(b) Loss Formulation No residuals 79.0 0.37 55.0% Without residual correction, weak noise corrupts model (+6.6 RMSE)
(c) Backbone ViT-S backbone 72.5 0.47 51.6% Matches ResNet-50 performance
(c) Backbone Swin-S backbone 71.0 0.49 50.6% Hierarchical tokens capture multi-scale canopy texture

Key Findings

  • Detection breakdown in dense satellite regimes: Off-the-shelf object detectors suffer catastrophic performance collapses on satellite tree counts. In Rwanda, YOLOv8 yields an RMSE of 448.7 with negative correlation (\(R^2 = -0.50\)), while CenterNet deteriorates to \(R^2 = 0.01\). Bounding box assumptions fail when individual tree crowns interlock and dissolve into continuous textures.
  • Critical impact of unbalanced transport and residual correction: Replacing UOT with balanced OT increases Rwanda RMSE from 72.4 to 78.3, and removing marginal residual correction causes a further drop to 79.0 RMSE. Dynamic residual correction allows the model to cleanly exploit 215M noisy LiDAR labels without directly overfitting erroneous annotations.
  • Cross-region transferability: Models trained on high-resolution Gaofen-2 (China, 0.8 m) generalize effectively to European SPOT-6 images (76.9% nMAE). Joint training across all three regions achieves 49.3% average nMAE, establishing the viability of multi-continent satellite foundation models for tree counting.

Highlights & Insights

  • Transport residuals as a self-supervised error detector: The formulation reveals that marginal deficits in the transport plan (\(\gamma^\top \mathbf{1} - y\)) serve as a direct, parameter-free spatial estimator for false-positive and false-negative annotations.
  • Continuous measure matching over discrete bounding boxes: Shifting from discrete instance detection to continuous density transport provides an elegant mathematical foundation for remote sensing tasks operating near or below the sensor point-spread function limit.
  • Stratified supervision decoupling: Pairing global \(L_1\) count anchors on sparse manual data with UOT density matching on dense weak data offers a general template for multi-fidelity remote sensing tasks.

Limitations & Future Work

  • Quadratic complexity of exact Sinkhorn iterations: The memory and compute scaling of the optimal transport plan is \(\mathcal{O}(N^2)\) in pixel count, restricting the training patch resolution to \(64 \times 64\) and limiting wide-area landscape context.
  • Absence of 3D allometric priors: The current formulation operates purely on 2D spectral densities without integrating canopy height models into allometric scaling laws (e.g., self-thinning rule and crown-diameter-to-height scaling), which could provide physical constraints for biomass estimation.
  • vs DM-Count (NeurIPS 2020): DM-Count introduced balanced OT for crowd counting, enforcing exact total probability conservation. In satellite forestry, noisy over- and under-counting cause balanced OT to produce over-confident artifacts; TreeMatch's UOT relaxation with residual bootstrapping systematically handles this discrepancy.
  • vs TreeFormer (ISPRS 2023): TreeFormer applies localized window self-attention for density regression but lacks geometric measure transportation, resulting in diffuse and oversmoothed tree distributions in dense canopies.

Rating

  • Novelty: โญโญโญโญโญ (Pioneering integration of unbalanced optimal transport and marginal residual bootstrapping for satellite-scale tree counting)
  • Experimental Thoroughness: โญโญโญโญโญ (Comprehensive validation spanning three continents, three satellite sensors, and 25k \(\text{km}^2\) with thorough cross-region ablations)
  • Writing Quality: โญโญโญโญโญ (Clean mathematical formulation, well-structured arguments, and transparent baseline benchmarking)
  • Value: โญโญโญโญโญ (Publicly provides the TinyTrees benchmark and TreeMatch code, establishing an essential tool for planetary carbon and biodiversity monitoring)