title: >- [Paper Note] Dual-Margin Embedding for Fine-Grained Long-Tailed Plant Taxonomy description: >- [ECCV 2026][Fine-Grained][Dual-Margin Embedding] Reshaping decision boundaries with dual margins and norm-guided sampling to solve fine-grained, long-tailed, and open-set plant taxonomy. tags: - ECCV 2026 - Fine-Grained Recognition - Long-Tailed Learning - Dual-Margin Loss - Plant Taxonomy date: 2026-09-19 content_hash: 01c30bf909c4e777

Dual-Margin Embedding for Fine-Grained Long-Tailed Plant Taxonomy¶

Conference: ECCV 2026
Paper: ECCV Official Link
Area: Fine-Grained Recognition / Long-Tailed Learning / Plant Taxonomy
Keywords: Plant Taxonomy, Fine-Grained Recognition, Long-Tailed Learning, Dual-Margin Embedding, Prototype Alignment

TL;DR¶

Addressing the co-occurring challenges of fine-grained morphological ambiguity, extreme long-tailed taxa distribution, and open-world domain shifts, this paper proposes TaxoNet, which couples a mathematically grounded dual-margin objective to suppress repulsive head-class gradients on rare prototypes with an adaptive norm-guided sample selection strategy.

Background & Motivation¶

Biodiversity conservation underpins planetary resilience and climate regulation, making automated ecological taxonomy through computer vision a core technology for scalable biodiversity monitoring. However, practical deployment in wild environments faces intertwined, compounding difficulties. Real-world ecological observations follow an acute long-tailed distribution where a few common (head) taxa dominate data collections, while rare, endemic, and endangered taxa (tail) suffer from extreme data scarcity. Concurrently, plant taxonomy requires fine-grained visual discrimination across species within identical genera that share subtle morphological traits (e.g., Acer rubrum vs. Acer saccharum), whereas phenotypic variations caused by seasonal phenology, growth stages, and localized viewing angles produce high intra-class variance.

Conventional long-tailed learning paradigms—including data resampling, loss reweighting (CBL), logit adjustments (LA), and label-distribution-aware margin loss (LDAM)—predominantly target closed-set scenarios and lack explicit metric mechanisms to reinforce intra-class compactness and inter-class angular margins. Under standard cross-entropy or additive margin softmax classifiers (e.g., AM-Softmax), optimization dynamics suffer from severe "prototype misalignment": dominant head-class instances exert pervasive, cumulative repulsive forces on tail-class prototypes whenever rare taxa act as competing non-targets. This constant drift pushes tail prototypes away from their actual class-conditional embedding centers, undermining representation geometry. Furthermore, real-world field applications are subject to substantial spatiotemporal distribution shifts across habitats and require reliable rejection of previously undocumented taxa in open-set regimes.

This paper tackles these challenges directly from the perspective of embedding geometry and gradient dynamics. By explicitly reformulating decision boundaries to offset head-class gradient repulsion on tail-class representations, the method preserves the metric structure of rare taxa while leveraging embedding norms as confidence and diversity indicators. Core idea: Introduce a dual-margin penalization loss that simultaneously enhances target class attraction and suppresses non-target class repulsion, paired with norm-guided sample selection to stabilize rare prototype alignment and boost generalization under open-world ecological conditions.

Method¶

Overall Architecture¶

TaxoNet employs a ResNet-101 backbone network $\phi(\cdot)$ pretrained on ImageNet to map plant images into $d$-dimensional normalized embedding representations $\hat{\mathbf{x}}$, measuring cosine similarity logits against learnable class prototype vectors $\hat{\mathbf{w}}_j$ distributed on a hypersphere. The overall pipeline coordinates two interdependent mechanisms: at the input level, an adaptive sample selection pipeline combines AugMix augmentation with embedding norm ranking to selectively feed scarce and high-variance instances into training batches; at the objective level, a class-prior-adjusted dual-margin loss adaptively alters both target additive margins and non-target penalization margins to prevent prototype drift caused by dominant classes.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Input Plant Image Batch"] --> B["Norm-Guided Sample Selection<br/>AugMix tail oversampling + low-norm filtering"]
    B --> C["Backbone Feature Extraction<br/>ResNet-101 hyperspherical mapping"]
    C --> D["Dual-Margin Penalization Loss<br/>Target attraction + non-target repulsion suppression"]
    D --> E["Class-Relative Margin Regularization<br/>Prior-guided power smooth constraint"]
    E --> F["Fine-Grained Long-Tailed Classification & Open-Set Taxa Rejection"]

Key Designs¶

1. Dual-Margin Penalization Loss: Symmetrically Reshaping Target and Non-Target Logits To resolve prototype misalignment where dominant head classes continuously repel tail prototypes in the softmax denominator, TaxoNet incorporates a dual-margin constraint into hyperspherical embedding learning: $$ \mathcal{L}{\mathrm{ours}} = -\log \frac{e^{s(z_y - m_y)}}{e^{s(z_y - m_y)} + \sum $$ where } e^{s(z_j - m_j)}$z_j = \hat{\mathbf{w}}_j^\top \hat{\mathbf{x}}$ denotes the cosine logit, and $s$ is fixed to 32.0 to avoid gradient scale divergence under long-tailed imbalance. For ground-truth class $y$, the target margin is configured as $m_y = m_0 - \Delta_y$; for any competing non-target class $j \neq y$, the non-target margin is set to $m_j = \Delta_j$. Here, $m_0$ is a pre-configured base margin, while $\Delta_c$ represents an adaptive class-relative adjustment. When a head-class instance is processed, rare tail classes frequently act as non-targets; because rare classes are assigned larger $\Delta_j$, the effective logit $z_j - m_j$ in the denominator is substantially reduced. Theoretical proof demonstrates that the head-class repulsive gradient norm on tail prototype $\hat{\mathbf{w}}_c$ satisfies $\|\frac{\partial \mathcal{L}_i}{\partial \hat{\mathbf{w}}_c}\| \leq \exp(m_{y_i} - m_c)$, exponentially dampening destructive prototype displacement and preserving convergence toward the class-conditional mean $\boldsymbol{\mu}_c$.

2. Class-Relative Margin Regularization: Adaptive and Stable Prior Alignment Rather than imposing rigid, handcrafted class margins that could destabilize optimization, class-relative margins $\Delta_c$ are parameterized through a monotonic smooth function tied to empirical label frequencies: $$ \Delta_c = m_0 \cdot \left( \frac{\rho_c}{m_0} \right)^{\zeta(\gamma)} $$ where $\rho_c \propto -\log \frac{N_c}{N}$ represents the normalized log-frequency class statistic normalized to $\sum_c \rho_c = m_0$, with $N_c$ denoting sample count in class $c$. The learnable parameter $\gamma$ undergoes smooth transformation via $\zeta(\gamma) = \log(1 + e^\gamma)$. To ensure stable convergence, a power-based regularization term penalizes deviations from empirical class priors: $$ \mathcal{L}{\mathrm{reg}} = \sum (\Delta_c - \rho_c)^2 $$ The composite objective is given by }$\mathcal{L} = \mathcal{L}_{\mathrm{ours}} + \lambda \mathcal{L}_{\mathrm{reg}}$. Consequently, practitioners need only specify base margin $m_0$ (set to 0.15), allowing class-specific margin geometries to adapt automatically.

3. Norm-Guided Sample Selection: Exploiting Phenotypic Variance and Tail Diversity Standard random oversampling of scarce tail classes typically leads to model memorization and overfitting on limited specimen appearances. TaxoNet leverages the geometric correlation between unnormalized embedding norms and prediction confidence in hyperspherical metric learning: low-norm instances signify either rare tail classes or high-variance samples driven by morphological variations (e.g., leafless winter twigs, partial floral views, or altered canopy structures). In each batch of $B$ training images, TaxoNet stochastically samples an additional $b=8$ tail-class candidates with AugMix transformations at Bernoulli probability $p=0.1$. From the resulting $B+b$ candidates, the network ranks unnormalized feature norms and retains only the $B$ lowest-norm instances for backpropagation. As the network masters these challenging representations over training epochs, their embedding norms expand, adaptively transitioning priority to other under-learned instances without redundant memorization.

Loss & Training¶

The overall training objective combines the dual-margin softmax and prior regularization: $$ \mathcal{L} = -\log \frac{e^{s(\hat{\mathbf{w}}y^\top \hat{\mathbf{x}} - (m_0 - \Delta_y))}}{e^{s(\hat{\mathbf{w}}_y^\top \hat{\mathbf{x}} - (m_0 - \Delta_y))} + \sum} e^{s(\hat{\mathbf{w}j^\top \hat{\mathbf{x}} - \Delta_j)}} + \lambda \sum (\Delta_c - \rho_c)^2 $$ All experiments utilize an ImageNet-pretrained ResNet-101 backbone with input images standardized to }$512 \times 512 \times 3$. Key hyperparameters are configured as follows: base margin $m_0 = 0.15$, scaling factor $s = 32.0$, batch size $B = 32$, oversampled pool buffer $b = 8$, and AugMix selection probability $p = 0.1$.

Key Experimental Results¶

Main Results¶

TaxoNet is evaluated across three diverse benchmarks covering urban street trees, worldwide natural observations, and herbarium collections: Google Auto-Arborist (comprising regional sub-datasets AA-Central, AA-West, and AA-East), iNat-Plantae (Plantae kingdom observations from iNaturalist-2019), and NAFlora-Mini (herbarium specimens). Performance is benchmarked using Rank-1 accuracy (R@1) alongside macro-averaged recall (Recall, %) to capture unbiased recognition across both common and rare taxa.

Dataset	Metric	TaxoNet (Ours)	LDAM (Prev. SOTA)	Gain / Difference
Google AA-Central	R@1 (%)	91.92	92.31	-0.39
Google AA-Central	Recall (%)	72.90	67.85	+5.05
Google AA-West	R@1 (%)	85.94	85.96	-0.02
Google AA-West	Recall (%)	67.67	62.82	+4.85
Google AA-East	R@1 (%)	84.32	85.32	-1.00
Google AA-East	Recall (%)	64.96	62.21	+2.75
iNat-Plantae	R@1 = Recall (%)	83.21	81.57	+1.64
NAFlora-Mini	R@1 (%)	91.52	91.26	+0.26
NAFlora-Mini	Recall (%)	90.40	89.96	+0.44

Ablation Study¶

Ablation experiments evaluate the cumulative contribution of each component across AA-Central, AA-West, and iNat-Plantae under identical backbone architectures:

Config	AA-Central Recall	AA-West Recall	iNat-Plantae Recall	Note
TaxoNet w/ A, C	63.91	60.81	78.91	Base margin $m_0$ only, no oversampling
TaxoNet w/ B, C	67.59	63.15	80.01	Dual-margin mechanism, no oversampling
TaxoNet w/ B, D	67.56	63.12	81.08	Dual-margin + random oversampling
TaxoNet w/ B, E	69.48	65.49	81.87	Dual-margin + norm-guided oversampling
TaxoNet w/ B, E, F (Full)	72.90	67.67	83.21	Dual-margin + norm-guided sampling + regularization

Note: A: Base Margin Only; B: Dual-Margin; C: No Oversampling; D: Oversampling with Random Selection; E: Oversampling with Norm-Guided Selection; F: Regularization.

In open-set recognition (calibrated at 95% TPR with 88 novel unseen taxa comprising 8,845 test instances), TaxoNet achieves 91.28% TNR and 92.26% overall accuracy on AA-Central, and 89.84% TNR and 90.38% overall accuracy on AA-West, significantly outperforming LDAM (88.59% TNR on AA-Central; 87.09% TNR on AA-West) and CBL (86.34% TNR on AA-Central).

Key Findings¶

The dual-margin mechanism is the primary driver of tail-taxa performance gains. Moving from base margin alone to the dual-margin formulation elevates macro-recall on AA-Central from 63.91% to 67.59%, directly corroborating the theoretical proof regarding suppressed repulsive gradients.
Norm-guided selection is substantially superior to naïve random oversampling. While random oversampling causes performance to stagnate or slightly fluctuate (67.59% vs. 67.56%), norm-guided selection lifts recall to 69.48%, proving that prioritizing geometrically low-norm instances alleviates memorization pitfalls.
Prior-based power regularization provides essential training stability, unlocking a further leap to 72.90% macro-recall on AA-Central.
Benchmarked against multimodal large language models (MLLMs like GPT-4o and Gemini-2.5-Flash) and foundation vision models (BioCLIP), generalist MLLMs with botanical expert persona prompting and chain-of-thought (CoT) reasoning struggle on species-level discrimination (achieving only 32%~41% macro-recall). TaxoNet achieves 83.21% species-level recall—outperforming GPT-4o by more than 2x—highlighting the indispensable nature of specialized metric geometries for fine-grained ecological observation.

Highlights & Insights¶

Theoretically Rigorous Dual-Margin Formulation: Extends hyperspherical additive margin losses from face verification to ecological long-tailed recognition by proving that non-target margins exponentially limit the repulsive gradient impact of head samples on tail-class prototypes, preventing prototype misalignment.
Norm as an Unsupervised Confidence and Variance Indicator: Elegantly exploits the intrinsic property of margin-softmax embeddings—where lower feature norms reflect classification uncertainty and morphological heterogeneity—to dynamically select hard cases and rare taxa without manual heuristic rules.
Superior Domain Generalization and Novel Taxa Rejection: By enforcing tight intra-class clustering and wide angular inter-class margins, TaxoNet models trained on regional data (AA-Central) generalize better across geographic distribution shifts (AA-West, AA-East) and achieve over 91% TNR on unseen novel species.

Limitations & Future Work¶

Visual Ambiguity Under Extreme Context Scarcity: Misclassifications still occur when specimens lack contextual organs (e.g., close-up images containing only a isolated leaf or flower petal without visible bark, fruit, or branching structure), such as confusing Opuntia polyacantha with the congeneric Opuntia cespitosa, or Cornus amomum with Persicaria chinensis.
Future Improvements: Hyperparameters governing norm selection (oversampling buffer $b$ and Bernoulli probability $p$) are currently static; implementing an adaptive schedule based on learning progress is a promising next step. Furthermore, integrating multi-view vision-language prompts could help resolve visual ambiguities among congeneric taxa.

vs. LDAM (Label-Distribution-Aware Margin Loss): LDAM enforces margin penalties strictly on the target class logit based on sample frequencies; TaxoNet introduces an explicit dual-margin mechanism on non-target classes to curb head-class gradient repulsion, complemented by geometric norm-guided sampling.
vs. AM-Softmax / ArcFace: Standard angular margin losses assume uniform label distributions for balanced identification; TaxoNet adapts hyperspherical geometry to severe long-tailed ecological spectra by aligning prototypes with true class means.
vs. BioCLIP / GPT-4o: While foundation vision-language models demonstrate broad semantic retrieval at coarse genus ranks, they falter at fine-grained, tail species discrimination; TaxoNet demonstrates that specialized metric learning frameworks remain vastly superior for high-precision botanical taxonomy.

Rating¶

Novelty: ⭐⭐⭐⭐☆ Elegant dual-margin gradient mitigation combined with norm-guided variance sampling for ecological taxonomy.
Experimental Thoroughness: ⭐⭐⭐⭐⭐ Rigorous validation across urban, natural, and herbarium benchmarks against long-tailed baselines and foundation MLLMs.
Writing Quality: ⭐⭐⭐⭐⭐ Coherent narrative, clear mathematical proofs, and well-structured empirical ablations.
Value: ⭐⭐⭐⭐⭐ Delivers an actionable, high-performance algorithm for automated biodiversity observation and open-world ecological AI deployment.