Purify then Guide: Rethinking Domain Generalization for Multimodal Face Anti-Spoofing¶

Conference: ECCV 2026
Paper: ECCV 2026
Code: https://github.com/murInJ/MMDA
Area: Human Understanding
Keywords: Face Anti-Spoofing; Multimodal; Domain Generalization; CLIP; Differential Attention

TL;DR¶

This paper recasts multimodal domain-generalized face anti-spoofing (FAS) as "purify then guide": MD2A, a same-domain cross-sample differential attention module, subtracts the domain- and modality-shared residual from the fused features; RS2 then softly aligns the purified RGB/IR/DEPTH features to real/spoof subspaces defined by CLIP text prompts, while a U-shaped dual-space adaptation (U-DSA) adds task capacity without destroying the pretrained geometry — lowering average HTER from 13.63% to 4.00% on WMCA/CeFA/PADISI/SURF.

Background & Motivation¶

Face recognition already underpins high-stakes applications such as payment, access control and surveillance, which makes face anti-spoofing (FAS) a necessary component. To make the decision more reliable, recent systems increasingly capture RGB, infrared (IR) and depth (DEPTH) streams at once: appearance, geometry and material cues are complementary and do separate live faces from attack artefacts better than any single modality. Yet cross-domain generalization of multimodal FAS remains poor — performance drops sharply under a new camera, a new environment or a new attack type. Multimodal cross-domain settings also carry two coupled shifts that the unimodal case does not have. First, cross-modal relations themselves (how complementary, how reliable, what scale RGB and IR have relative to each other) are domain-dependent, so the fused representation drifts across groups. Second, learning a new decision boundary by fine-tuning on limited source domains yields an overly smoothed separator in the joint multimodal space, which leaves hard target samples sitting near the boundary (Fig. 1(b) of the paper).

Most existing methods attack this directly in the noisy fused space, either by aligning features or by relearning a boundary: MMDG analyzes the conflict between the domain-generalization objective and multimodal fusion and mitigates it with uncertainty rectification and modality re-balancing, while DADM aligns domains and modalities simultaneously and is the strongest multimodal baseline to date. The problem is that in a fused feature the spoof-relevant cues and the domain/modality artefacts are entangled, so aligning it forces useful and useless patterns to match at the same time. Two diagnostics in the paper support this reading. An ANOVA-style ratio \(r=\text{Var}_{between}/\text{Var}_{total}\) over per-class embeddings shows that cross-domain variation, especially for bonafide samples, has a strong mean-shift component — i.e. an approximately additive bias. And same-domain pairs are significantly closer than cross-domain pairs under both \(\ell_2\) and cosine distance (with separated bootstrap confidence intervals), so local neighborhoods are relatively stable within a group. If intra-group neighborhoods are stable and cross-group discrepancies are mostly a shift, then a non-trivial part of that discrepancy is a group-shared residual that can be estimated under same-group conditioning and removed before any alignment, rather than being absorbed by the alignment step.

The paper therefore views a fused feature as the sum of group-invariant semantics and a group-dependent bias, purifies the bias away first, and then guides the clean representation into a stable real/spoof structure. The carrier of the first step is MD2A (Modality-Domain Joint Differential Attention); the second step is shared by RS2 (Representation Space Soft alignment), which performs the guiding, and U-DSA (U-shaped Dual Space Adaptation), which deepens task adaptation without damaging CLIP's pretrained geometry. Core idea: split multimodal domain-generalized FAS into "purify then guide" — use same-domain cross-sample differential attention to estimate and subtract the domain/modality-shared residual, then softly align the purified features to the real/spoof subspace spanned by multiple prompts in CLIP's text space, and use a U-shaped dual-space adaptation so that task adaptation does not destroy the pretrained geometry.

Method¶

Overall Architecture¶

MMDA takes a batch of RGB/IR/DEPTH images together with a set of captions describing real and spoof faces, and outputs a live/spoof decision per sample. The three image streams and the text go through CLIP's image and text encoders respectively; the text encoder stays frozen throughout, while the image encoder is fine-tuned on the FAS data together with the new modules. The multimodal visual embeddings are first fused and denoised by MD2A — the "purify" step, which does not alter the backbone but subtracts a residual from the fused representation. The representation then enters U-DSA, which walks down a bottom path of layer-wise, parameter-efficient adaptation (each layer is an MoE) and then walks back up a feedback path that remaps deep representations into shallower spaces, while RS2 acts as a layer-wise soft constraint pulling each layer's representation toward the text-defined real/spoof subspace — together, the "guide" step. Finally, a shallow shared classifier produces the decision from the adapted visual embeddings.

One clarification about the framework diagram below: RS2 and U-DSA are drawn one after the other so that the figure matches the order of the key designs. In the implementation RS2 is not a separate forward module but a layer-wise loss constraint applied to the intermediate representations of U-DSA.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["RGB / IR / DEPTH + captions"] --> B["CLIP image encoding + multimodal fusion"]
    B --> C["MD2A<br/>same-domain cross-sample denoising"]
    C --> D["RS2<br/>align to real/spoof text subspace"]
    D --> E["U-DSA<br/>deep adaptation + shallow remap"]
    E --> F["shared classifier → live / spoof"]

Key Designs¶

1. MD2A: let the noise branch of differential attention look at another sample from the same domain

Differential Attention (DA) was originally used by running two attention branches over two views of the same sample and subtracting their responses, which suppresses sample-wise spurious patterns. MD2A changes exactly one thing: the noise branch is no longer fed another view of the same sample but the multimodal features of a different sample from the same domain. Concretely, let \(X=\{x_0,\dots,x_b\}\) be the fused multimodal features in a mini-batch and \(D\) the corresponding domain labels. For each \(x_i\) a same-domain sample \(\tilde x_i\) is picked at random (requiring \(\tilde x_i \neq x_i\) whenever another same-domain sample exists, and falling back to \(\tilde x_i = x_i\) otherwise); the pair \((x_i,\tilde x_i)\) is concatenated along the feature dimension, projected, and split into a main branch \((Q,K)\) and a differential branch \((Q',K')\), giving two attention maps \(A\) and \(A'\). The purified output is

\[X_{de} = \big[\text{softmax}(A) - \lambda\,\text{softmax}(A')\big]V\]

where \(\lambda\) follows the parameterization of the original differential attention (learnable vectors, layer-dependent; the paper sets \(l=14\) — the equation is corrupted in the cached text, ⚠️ refer to the original paper for its exact form). The reason this works lies in what the two branches share and where they differ: they share the same domain condition and the same modality configuration and differ only in identity and attack content, so their differential response predominantly captures domain- and modality-specific components rather than real-versus-spoof semantics. Treating that response as a residual and subtracting it removes the same-domain common-mode component over the whole batch, leaving a representation in which spoof-discriminative cues stand out more clearly. When \(\tilde x_i = x_i\) for all \(i\), MD2A degenerates to the original differential attention, so it strictly contains the old method as a special case. The paper also claims to show theoretically that MD2A tightens the generalization error bound in multimodal FAS (the proof is not developed in the main text, ⚠️ see the original paper/appendix).

2. RS2: align to a subspace spanned by multiple prompts, not to a single prompt

After purification the representation has to be guided into a stable real/spoof structure, and the choice of alignment target matters: defining the target from limited source-domain data re-encodes the very biases we want to remove, while hard-aligning visual features to a single text embedding collapses the rich spoof manifold onto one point. RS2 therefore moves the target from a point to a subspace. Given a set of captions, the frozen CLIP text encoder yields a set of text embeddings \(T\); each visual embedding \(v_i\) is only required to lie close to the region spanned by text embeddings of its own class, measured by the minimum cosine distance to same-class text embeddings:

\[d_i = \min_{t_j \in T(\hat y_i)} \frac{1}{2}\Big(1 - \frac{v_i \cdot t_j}{\lVert v_i \rVert\,\lVert t_j \rVert}\Big)\]

A smoothed binary cross-entropy is then applied to this distance-based score: labels are first smoothed with \(\epsilon=0.2\) (the true class takes \(1-\epsilon\), the other class \(\epsilon\)) to avoid over-confident targets, and a shared binary classifier is attached to both visual and text embeddings so that both sides receive discriminative supervision. The two terms add up to \(\mathcal{L}_{\text{RS2}}=\mathcal{L}_{\text{cls}}+\mathcal{L}_{\text{align}}\). The benefit runs both ways: the min operator lets a feature land anywhere reasonable inside the class subspace instead of sacrificing fine-grained cues to match one prompt, while the shared classifier brings the text side into the discriminative supervision rather than leaving it as a passive anchor.

3. U-DSA: adapt deep, remap shallow — without wrecking CLIP's geometry

RS2 presupposes that CLIP's representation space still has a meaningful structure, yet task adaptation on limited FAS data is exactly what distorts it: without deep adaptation the feature extractor lacks task-specific capacity, while aggressively fine-tuning deep layers warps the pretrained space and drags its decision boundary along. U-DSA resolves the tension with two moves: it applies RS2 layer-wise, and it builds a U-shaped dual-space pathway that carries deep adapted features back to shallower, more domain-invariant layers. The downward branch adapts layer by layer, \(v_i=\text{Adapt}_i(v_{i-1})\), where each \(\text{Adapt}_i\) is a parameter-efficient MoE; the feedback branch starts from the deepest representation and maps it back into the space of layer \(i\), \(v'_i = v_i + \text{Remap}_i(v'_{i+1})\), again with an MoE. The RS2 losses are applied to both \(v_i\) and \(v'_i\), which amounts to requiring every layer's representation to stay close to the text-defined real/spoof structure. As a result deep layers capture task-specific multimodal patterns while the feedback path and the layer-wise constraints keep the overall representation anchored near the pretrained geometry, and the final classifier operates in a space that is simultaneously enriched by adaptation and anchored by the original real/spoof separation. The experiments corroborate this asymmetric design: as U-DSA depth grows to 7 layers the AUC distribution shifts toward higher values (deep adaptation does help), but across all depths from 1 to 7 the best-performing layer is predominantly shallow or mid-depth, and the deepest layer never achieves the best AUC.

A Worked Example: one mini-batch of purification and guidance¶

Consider a mini-batch of 576 multimodal samples drawn from three source domains. For a sample \(x_i\) from domain A, MD2A looks inside the same batch for another domain-A sample \(x_j\), concatenates the two fused features, projects and splits them into the main and differential branches. Each branch computes one attention map; after subtraction, whatever \(x_i\)'s representation carried of "domain A's acquisition characteristics plus the inherent bias of this modality combination" is largely cancelled, leaving identity- and attack-related content. This is done at both training and inference time (the paper states explicitly that the concatenation pairing is applied in both, see the Limitations discussion). The purified representation then descends through 7 layers of MoE adaptation in U-DSA, being pulled by RS2 at every layer; the deepest representation is remapped back into shallower spaces along the feedback path, and the classifier produces the live/spoof decision on that brought-back, layer-wise-constrained representation.

Loss & Training¶

The training objective is the RS2 soft-alignment loss (a smoothed BCE distance term plus the shared-classifier discriminative term) accumulated layer by layer, together with the joint fine-tuning of the CLIP image encoder and the MD2A / U-DSA modules; the text encoder stays frozen. The backbone is CLIP ViT-B/16, inputs are resized to 224×224 and tokenized into 14×14 patch tokens with a 512-dim embedding; AdamW with learning rate 5×10⁻⁶, weight decay 1×10⁻³, batch size 576 and 100 epochs for all protocols; unless stated otherwise U-DSA uses 7 layers. Section 3 of the paper says the classifier operates on both visual and text embeddings, whereas Section 4.1 says it operates on the adapted visual embeddings — ⚠️ the two statements differ slightly, refer to the original implementation. The paper gives no explicit formula for "total loss = sum of per-layer RS2"; the description here summarizes Section 3.3 in words, ⚠️ weighting details refer to the original paper.

Key Experimental Results¶

Main Results¶

Experiments run on four multimodal FAS benchmarks — WMCA (W), CASIA-CeFA (C), PADISI (P) and CASIA-SURF (S) — under cross-dataset evaluation, reporting HTER (lower is better) and AUC (higher is better). Protocol 1 uses complete modalities for both training and testing under four leave-one-out domain splits (CPS→W means training on C/P/S and testing on W).

Method	CPS→W HTER	CPW→S HTER	CSW→P HTER	PSW→C HTER	Avg. HTER ↓	Avg. AUC ↑
FLIP (strongest unimodal DG)	13.19	11.73	17.39	22.14	16.11	90.83
MMDG	12.79	15.32	18.95	29.93	19.24	87.96
DADM (previous best)	11.71	6.92	19.03	16.87	13.63	92.96
CLIP (zero-shot-style baseline)	14.55	18.17	24.13	38.33	24.63	83.00
MMDA (Ours)	1.22	4.21	4.34	6.25	4.00	98.94

Protocol 2 adds test-time missing modalities (missing DEPTH / missing IR / missing both) on top of Protocol 1, and Protocol 3 uses only two datasets as source domains to simulate limited-source training.

Protocol	Test condition	Prev. best (DADM) HTER / AUC	MMDA HTER / AUC
Protocol 2	Missing DEPTH	21.56 / 85.17	11.10 / 93.97
Protocol 2	Missing IR	20.82 / 85.28	5.98 / 98.30
Protocol 2	Missing DEPTH + IR	22.61 / 84.04	13.36 / 93.74
Protocol 2	Average	21.66 / 84.83	10.14 / 95.33
Protocol 3	CW→PS	12.61 / 93.81	7.52 / 96.84
Protocol 3	PS→CW	20.40 / 89.51	6.30 / 98.35

Ablation Study¶

The MD2A ablation compares two adapter backbones (Dense and MoE) × four fusion-attention configurations on the PS→CW split; the RS2 ablation takes MoE + MD2A as its baseline and compares three alignment schemes.

Config	HTER ↓	AUC ↑	Note
Dense adaptor (baseline)	23.26	84.92	no extra attention
Dense + MHSA	25.85	82.95	standard multi-head self-attention actually hurts
Dense + DA	16.49	92.05	original differential attention
Dense + MD2A	13.47	94.20	same-domain cross-sample differential
MoE adaptor (baseline)	22.92	85.84	no extra attention
MoE + MHSA	12.83	93.25
MoE + DA	12.72	93.89
MoE + MD2A	9.70	95.23	best ablation configuration
MoE + MD2A, Vanilla Alignment	9.70	95.23	hard alignment to the matching text prompt
MoE + MD2A, Smooth Alignment	9.17	96.32	adds label smoothing
MoE + MD2A, RS2 Alignment (full)	8.88	97.20	smoothing + nearest text subspace + shared classifier

The paper also runs a pairing-swap invariance test to check whether MD2A performs common-mode suppression rather than arbitrary semantic subtraction: \(x\) is held fixed and only the paired sample \(\tilde x\) is swapped (same-domain / random / cross-domain), repeating the pairing 10 times per sample; it then measures the within-swap variance of the residual Var[R], the within-swap variance of the decision Var[z] (z is the spoof logit computed from the denoised feature), and the HSIC between features and domain labels (lower means weaker domain dependence).

Pairing strategy	Var[R]	Var[z]	Note
Same-domain	4.31×10⁻⁷	1.672×10⁻³	most stable residual
Random	7.48×10⁻⁷	1.662×10⁻³
Cross-domain	6.94×10⁻⁷	1.728×10⁻³	clear mean-residual drift (magnitude 9.55×10⁻⁷)

Key Findings¶

MD2A is the single largest contributor: on the MoE backbone it pushes HTER from 22.92% down to 9.70% (-13.22), and the gain clearly exceeds that of the original DA (12.72→9.70), which shows that the reformulation itself — feeding another same-domain sample into the noise branch — is what matters, not merely "using differential attention".
Notably, standard multi-head self-attention hurts on the Dense backbone (23.26→25.85): adding an attention module with no denoising inductive bias just fuses the domain/modality noise in as well.
RS2's gain over "hard alignment + smoothing" is only 0.29 in HTER, but AUC rises from 96.32 to 97.20 (+0.88), indicating that soft subspace alignment mainly improves the ranking quality of scores rather than simply lowering the error rate at a threshold; the paper reads this as evidence that shaping the geometry of CLIP's space is more effective than only updating weights or aligning to a single prompt.
On missing modalities, MMDA still reaches 5.98% HTER / 98.30% AUC when IR is missing, which the paper describes as very close to the complete-modality case — the purified RGB and DEPTH features plus RS2 alignment can compensate for the lost IR. Missing DEPTH degrades more clearly (11.10% / 93.97%), showing that the geometric information is harder to recover from RGB and IR alone.
The pairing-swap test supports the common-mode suppression account: same-domain pairing gives the smallest residual variance, switching to cross-domain pairing introduces a systematic 9.55×10⁻⁷ drift in the mean residual, and after subtraction the HSIC between features and domain labels drops from 1.596 to 0.269 (a drop of 1.327) while Var[z] stays at the 10⁻³ level, so denoising does not destabilize the decision itself.
In t-SNE, the fine-tuned CLIP baseline forms several overlapping clusters with limited cross-domain mixing within each class, whereas MMDA separates real from spoof much more cleanly with better cross-domain mixing inside each class; along U-DSA, samples from the input to the remapped deep output gradually concentrate into tighter clusters, indicating that adaptation refines features without drifting away from a stable decision structure.

Highlights & Insights¶

Swapping differential attention's "two views of one sample" for "another sample from the same domain" is a tiny change that upgrades sample-level noise suppression into group-level common-mode residual suppression. The idea transfers to any task with strong batch effects — multi-center medical imaging, cross-sensor detection, cross-subject physiological modeling — as long as a mini-batch can still supply same-group companions to estimate and subtract the common mode.
Validating the denoising mechanism with "swap the paired sample + three invariance metrics" instead of only reporting the HTER gain is the most instructive piece of experiment design here: Var[R] checks whether the estimate is stable, Var[z] whether the decision is disturbed, and HSIC whether domain dependence really drops. Together they separate "the mechanism runs as intended" from "the number went up".
Extending the alignment target from a point to a subspace (minimum distance to same-class text embeddings) essentially grants multimodal semantics a legal amount of ambiguity: real faces are expressed in many ways, and pinning them to one prompt penalizes that diversity as error. Any fine-grained binary classification on top of CLIP can reuse this trick directly.
The U-shaped structure yields a reusable empirical rule — "adapt deep, decide shallow": the paper's Fig. 4 shows that across total depths from 1 to 7 the best-performing layer is almost always shallow or mid-depth and the deepest layer is never best. Worth trying first in any "fine-tune a pretrained model + need cross-domain generalization" setting.

Limitations & Future Work¶

The authors acknowledge that MoE layers can be optimization-sensitive in pretrained spaces; this should show up as training instability (the paper reports no training curves or run-to-run variance), and future work targets more stable and adaptive dual-space adaptation.
A hidden implementation constraint: MD2A's pairing (Concat) is applied at both training and inference, which means a single test sample cannot be evaluated independently — one must assemble a batch containing same-domain companions, which in turn requires knowing test-time domain labels. The paper does not say where those labels come from or how the pairing pool is maintained, and it reports no degradation when the batch (and thus the supply of same-domain companions) shrinks. ⚠️ This is a real constraint for deployment, especially for streaming single-frame verification.
Protocol 3 only covers CW→PS and PS→CW, so the number of source domains is only probed at "2 sources" and "3 sources"; more extreme shifts such as unseen attack types or ethnicities are not analyzed separately.
The paper claims MD2A "theoretically tightens the generalization error bound" but the main text gives only the conclusion, with no assumptions or proof sketch. ⚠️ Refer to the original paper/appendix; readers should not treat this as evidence that the method comes with a theoretical guarantee.
The ablation grid is 2 adapters × 4 attention/alignment variants, and it lacks an end-to-end comparison with the pairing strategy (same-domain vs random vs cross-domain) as a training variable — Table 6 only swaps pairings at evaluation time, so how much performance is lost by training with random pairing is not reported.

vs MMDG (CVPR 2024): MMDG analyzes the conflict between the domain-generalization objective and multimodal fusion and mitigates it with uncertainty rectification and modality re-balancing at the fused-feature level; this paper argues the root cause is that semantics and artefacts are entangled in the fused feature, so it subtracts the domain/modality-shared residual first and aligns afterwards. The resulting average HTER is 4.00% versus 19.24%, but note that MMDG does not use CLIP pretraining, so part of that gap comes from the backbone.
vs DADM: DADM aligns domains and modalities jointly and is the strongest baseline before this work; this paper follows its protocol and cuts HTER on the hardest PS→CW split from 20.40% to 6.30%, without any modality-specific dropout or retraining for missing modalities.
vs Differential Transformer: the original differential attention differentiates two views of the same sample to suppress sample-wise noise; MD2A replaces the noise branch with another same-domain sample, lifting "sample-level" to "group-level" while retaining the original as the \(\tilde x_i = x_i\) degenerate case.
vs CLIP-based vision-language FAS such as FLIP: those methods mainly care about how to update or regularize pretrained weights and prompts, and are mostly unimodal or rely on simple late fusion; this paper focuses on how the structure of the pretrained representation space itself can be exploited and preserved, and extends that idea to multimodal fusion and missing-modality settings.

Rating¶

Novelty: ⭐⭐⭐⭐ The same-domain-cross-sample pairing is a small edit in itself, but it moves differential attention from the sample level to the group level and organizes "purify first, then align" into a clear principle; soft subspace alignment is a reasonable extension of existing ideas rather than a first.
Experimental Thoroughness: ⭐⭐⭐⭐ Four datasets, three protocol families (complete modality / missing modality / limited source) plus a mechanism-level pairing-swap analysis is above average for the FAS literature; the deductions are the missing end-to-end ablation on training-time pairing and any deployment-relevant batch-size sensitivity analysis.
Writing Quality: ⭐⭐⭐⭐ The motivation chain is backed by diagnostics (variance ratio + same-domain neighborhood stability) and the method is described clearly; however, several formulas are garbled in the text (partly a typesetting/OCR issue) and Sections 3 and 4.1 disagree on what the classifier consumes, so readers should watch for that.
Value: ⭐⭐⭐⭐ An average HTER drop from 13.63% to 4.00% and +5.98 AUC in cross-domain evaluation is a large margin, and the two lessons — "purify then guide" and "adapt deep, decide shallow" — transfer directly to other multimodal generalization tasks.