Sparsity-Inducing Divergence Losses for Biometric Verification¶

Conference: ECCV2026
arXiv: 2606.31664
Code: None
Area: Audio & Speech / Face Recognition
Keywords: Face Verification, Speaker Verification, α-Divergence Losses, Margin-Penalty, Sparse Posterior

TL;DR¶

Q-Margin shifts the margin penalty from geometric logit modifications to a probabilistic reference measure in \(\alpha\)-divergence losses. While maintaining posterior sparsity, it consistently outperforms ArcFace/CosFace baselines in low-FAR scenarios on IJB-B/C and VoxCeleb, and reduces training throughput overhead from 27% to 5% via exact top-K truncation.

Background & Motivation¶

Face verification and speaker verification have been widely adopted in high-security authentication scenarios. Recently, margin-penalty-based softmax losses—such as CosFace (subtracting \(m\) from cosine similarity) and ArcFace (adding \(m\) to the angle)—have become the de facto standards for these tasks. They significantly pull apart inter-class distances and compress intra-class dispersion by imposing a geometric penalty on target class logits. However, these margins are heuristically applied numerical operations on logits, which are disconnected from the probabilistic nature of the loss function—essentially begging the question: "Why modify the logits rather than the loss function itself?"

Another independent line of progress is the \(\alpha\)-divergence loss proposed by Roulet et al., which replaces KL divergence with \(\alpha\)-divergence as a regularizer for probability mapping, unifying and generalizing cross-entropy. This framework allows a non-uniform reference measure \(q\) to encode arbitrary priors, and when \(\alpha > 1\), the \(\alpha\)-softargmax produces sparse posteriors (with exact zero probability for most classes). This is highly appealing for massive classification heads (often scaling to millions) in face/speaker recognition. However, prior applications only utilized \(q\) to handle class imbalance, and it has never been used to encode margin penalties.

The Key Challenge of this paper stems from this discrepancy: the geometric margin (ArcFace/CosFace) relies on the exponential normalization of softmax and cannot be naturally generalized to the \(\alpha\)-divergence framework, while \(\alpha\)-divergence losses, despite having a superior theoretical foundation and sparse posteriors, lack a margin mechanism. The Key Insight is that given that the reference measure \(q\) can encode arbitrary prior information, the margin penalty can be entirely encoded in \(q\) rather than in the logits. Core Idea: Present Q-Margin loss, which achieves a probabilistic margin by lowering the reference measure of the target class to \(q_y = \exp(-s \cdot m)\). This unifies geometric margin with information-theoretic loss design under the \(\alpha\)-divergence framework, naturally degrading Q-Margin to CosFace as \(\alpha \to 1\).

Method¶

Overall Architecture¶

The calculation flow of Q-Margin revolves around the interaction between \(\alpha\)-softargmax and the reference measure. After extracting embedded features, the backbone network computes cosine similarities with the weight vectors of all classes, multiplied by a scaling factor \(s\) to obtain logits \(\theta = s \cdot c\). Concurrently, the reference measure \(q\) depresses the value of the target class to \(\exp(-s \cdot m)\) (while others remain \(1\)). Then, the \(\alpha\)-softargmax with \(q\) solves for the threshold \(\tau^*\) via iterative bisection to obtain the sparse posterior \(p\), which is finally substituted into the \(\alpha\)-divergence loss to compute gradients.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Backbone extracts embeddings"] --> B["Cosine similarity with class centers"]
    B --> C["Scale to logits<br/>θ = s · c"]
    C --> D["α-softargmax<br/>(Iterative bisection to solve for τ*)"]
    D --> E["Sparse posterior p<br/>(α>1, most p_j = 0)"]
    E --> F["Compute loss<br/>l_f(θ, y; q)"]

    G["Reference measure q<br/>q_y = exp(−s·m)"] --> D
    H["Take top-K% logits<br/>(K=5)"] -.-> D

Key Designs¶

1. Q-Margin: Encoding probabilistic margin with the reference measure

Traditional methods directly perform geometric surgery on logits—CosFace subtracts \(m\) from the target cosine similarity, and ArcFace adds \(m\) to the angle. Q-Margin takes a completely different path: it does not touch the logits but modifies the reference measure \(q\) in the \(\alpha\)-divergence loss. For the true label \(y\) of a sample, \(q_y\) is set to \(\exp(-s \cdot m)\), while other \(q_j\) remain \(1\). The effect of this downweighted \(q_y\) in the \(\alpha\)-softargmax is equivalent to telling the model that "the prior of the target class is compressed," forcing the model to output a significantly higher target class logit to offset the penalty, thereby implicitly widening decision boundaries in the probability space. When \(\alpha \to 1\), \(\alpha\)-softargmax degenerates to standard softargmax, and taking the logarithm of the downweighted \(q_y\) is equivalent to \(\cos(\theta) - m\)—precisely recovering CosFace and providing it with a probabilistic explanation. The paper also attempts to jointly use the reference measure margin of Q-Margin with the geometric angle margin of ArcFace, but experiments yielded no additional gains, ultimately retaining the pure probabilistic form.

2. Sparse posterior of \(\alpha\)-divergence and joint tuning of \(\alpha\)-s

When \(\alpha > 1\), the posterior for each class in \(\alpha\)-softargmax is \(p_j = q_j [1 + (\alpha-1)(\theta_j - \tau^*)]_+^{1/(\alpha-1)}\), where \([\cdot]_+ = \max(0, \cdot)\) means classes below the threshold are assigned exactly zero probability. On face verification (WebFace42M, 2M+ classes), with \(\alpha=1.25\), only \(\sim 1,020\) classes (\(\sim 0.05\%\)) on average remain active. This sparsity forces the model to allocate probability mass only to the most likely few classes, securing sharper boundaries. There is a coupling between \(\alpha\) and the scale factor \(s\): both control the sharpness of softargmax (increasing \(\alpha\) sharpens the mapping, and increasing \(s\) has a similar effect), thus they cannot be tuned independently; the optimal \(s\) decreases as \(\alpha\) increases. For faces (2M classes), the optimal configuration is \(\alpha=1.25\) and \(s=32 \sim 35\); speaker verification (5,994 classes) requires higher sparsity due to fewer categories, where the optimal configuration is \(\alpha=1.5 \sim 1.75\) and \(s=5 \sim 10\).

3. Exact top-K truncation: Utilizing sparsity to feed back into efficiency

Since \(\tau^*\) of \(\alpha\)-softargmax needs to be solved via iterative bisection, full computation over 2M+ classes causes approximately a 27% drop in throughput. However, the sparse posterior provides an elegant solution: the paper proves that when the active support set is a subset of the top-K logits, calculating \(\tau^*\) solely on the top-K is exact rather than an approximation. Setting \(K=5\%\) (a conservative value, far higher than the maximum observed active rate of 0.4%), the throughput recovers from 1,466 samples/s to 1,900 samples/s, which is only 5% slower than ArcFace (2,005); further tightening to \(K=1\%\) almost fully restores it (1,978). The runtime also includes an integrity check to guarantee that the smallest retained logit itself is inactive—experimental results never triggered a truncation error.

Loss & Training¶

The loss function of Q-Margin is defined as:

\[Links: L_{\text{Q-M}} = l_f(s \cdot \mathbf{c}, \mathbf{y}; \mathbf{q})\]

where \(l_f\) is the \(\alpha\)-divergence Fenchel-Young loss, \(\mathbf{c}\) is the cosine similarity vector between embeddings and class centers, and the reference measure \(\mathbf{q}\) is defined as \(q_j = \exp(-s \cdot m \cdot \delta_{y,j})\). In training, SGD is used for faces with an initial \(lr = 0.1\), decaying in three stages over 20 epochs; speaker verification uses SGD with a linear warmup of 6 epochs followed by exponential decay.

Key Experimental Results¶

Main Results¶

Under controlled training data (WebFace42M) and backbone (ResNet-100) settings, Q-Margin comprehensively outperforms ArcFace, CosFace, EntMax, and SparseMax on stringent low-FAR metrics of IJB-B and IJB-C:

Method	Data	Backbone	IJB-B@1e-5	IJB-C@1e-4	IJB-C@1e-5
ArcFace	WF42M	R-100	92.26	97.54	95.70
CosFace	WF42M	R-100	93.15	97.49	95.82
EntMax (\(\alpha\)=1.25)	WF42M	R-100	93.06	97.55	96.11
SparseMax (\(\alpha\)=2)	WF42M	R-100	91.84	97.12	95.28
Q-Margin (\(\alpha\)=1.25, s=35)	WF42M	R-100	93.67	97.78	96.35

Speaker verification is equally competitive: Q-Margin reaches 74.60% TAR@1e-4 on VoxCeleb1-H (\(\alpha=1.75\)), whereas ArcFace is 72.08% and CosFace is 70.61%.

Ablation Study¶

Exact acceleration effects brought by sparse posteriors:

Configuration	Throughput (samples/s)	Relative to ArcFace
ArcFace	2005.13	1x (Baseline)
Q-Margin (Full 100% logits)	1466.45	0.73x (−27%)
Q-Margin (top 5%)	1900.26	0.95x (−5%)
Q-Margin (top 1%)	1978.07	0.99x (−1%)

Hyperparameter sensitivity (IJB-C@1e-5):

\(\alpha\)	s	m	Metric	Note
1.25	35	0.2	96.35	Best config for face
1.25	32	0.5	96.00	Performance drops as m increases
1.5	10	0.2	95.64	Increasing \(\alpha\) requires lowering s

Key Findings¶

The biggest advantage of Q-Margin is concentrated in low-FAR (\(10^{-4} / 10^{-5}\)) scenarios—precisely the operational points of interest for high-security verification (finance, border control), where ArcFace/CosFace are prone to performance bottlenecks.
\(\alpha\) and \(s\) are coupled and cannot be tuned independently. Rule of thumb: start with \(\alpha=1.25, m=0.2\) and gradually ramp up \(s\).
Posterior sparsity is inversely proportional to the number of classes: for million-scale categories, \(\alpha=1.25\) is sufficient, while thousand-scale categories require \(\alpha=1.5 \sim 1.75\).
Performance matches PartialFC on IJB-C (97.78 vs 97.82@1e-4), but Q-Margin does not rely on extra negative sampling strategies; its acceleration stems directly from its mathematical properties.

Highlights & Insights¶

Most significant theoretical contribution: Proving that the margin penalty can be entirely probabilistic—converting geometric surgery to reference measure modifications, making CosFace a limit case of Q-Margin as \(\alpha \to 1\), thus establishing an information-theoretic foundation for margin-penalty losses.
Win-win of sparse posteriors: The sparsity when \(\alpha > 1\) improves embedding discriminability on one hand, and is recursively utilized to accelerate the computation of \(\tau^*\) on the other—showing an elegant feedback loop where mathematical properties translate directly into engineering efficiency.
Cross-modal consistency: Q-Margin outperforms respective baselines on both face and speaker verification modalities, demonstrating that probabilistic margin is a generalizable framework rather than a face-specific trick.
Orthogonal to adaptive strategies like AdaFace and elastic margins—the probabilistic margin of Q-Margin can be integrated with dynamic sample difficulty adjustments in the future.

Limitations & Future Work¶

Q-Margin introduces an extra hyperparameter \(\alpha\) which is coupled with \(s\), leading to higher tuning costs compared to ArcFace/CosFace.
Most WebFace42M experiments were run only once (training on million-scale categories is computationally expensive) and lack confidence intervals, although the standard deviation of repeat experiments was only 0.07.
Controlled experiments only baseline against ArcFace/CosFace, and re-implementations of several SOTA methods (AdaFace, PartialFC) are not fully disclosed.
Future directions: dynamizing the reference measure (tuning \(q_y\) based on sample difficulty) and combining it with negative sampling strategies for large-scale categories like PartialFC.

vs ArcFace/CosFace: Geometric margin vs probabilistic margin. Q-Margin views CosFace as a special case when \(\alpha \to 1\), unifying them under a single formulation with a stronger theoretical basis, though the new hyperparameter \(\alpha\) increases the entry barrier.
vs AdaFace / ElasticFace: Adaptive margin methods scale margins within the softmax framework, while Q-Margin innovates at the underlying divergence level. They are orthogonal and can be combined.
vs PartialFC: Sparse posteriors and negative sampling reach the same goal through different paths—both significantly downscaling computation costs, matching performance on IJB-C. The difference is that PartialFC relies on heuristic sampling, while Q-Margin's acceleration is guaranteed by exact mathematical proof.
vs EntMax / SparseMax: SparseMax fixes \(\alpha = 2\) and lacks a probabilistic margin. Q-Margin offers higher flexibility, outperforming SparseMax in both face and speaker verification.

Rating¶

Novelty: ⭐⭐⭐⭐½ Encoding margin through reference measures is an elegant theoretical innovation, unifying two independent research lines.
Experimental Thoroughness: ⭐⭐⭐⭐½ Covers dual modalities (IJB-B/C + VoxCeleb), conducts rigorous hyperparameter sweeps, and provides complete throughput analysis; unfortunately, most WebFace42M experiments are single-run.
Writing Quality: ⭐⭐⭐⭐ Logically clear with mathematically rigorous derivations, though some sections are highly equation-dense.
Value: ⭐⭐⭐⭐½ Formulates a probabilistic theoretical foundation for margin-penalty losses, provides immediately applicable sparse acceleration tricks, and is highly inspiring for both the face and speaker verification communities.