TRAM: Fine-Tuning-Free Test-Time Adaptation with Few Bonafide Samples for Generalized Face Anti-Spoofing

Conference: ECCV 2026
Paper: ECCV Official
Area: Human Understanding
Keywords: Face Anti-Spoofing, Test-Time Adaptation, Fine-Tuning-Free, Relational Modeling, Prompt Learning

TL;DR

Addressing the scarcity of target-domain spoof samples and the prohibitive cost of model fine-tuning in generalized face anti-spoofing (FAS), TRAM introduces a fine-tuning-free test-time adaptation framework that pairs incoming test images with a few target-domain bonafide anchors and performs relative image-pair-text contrastive learning with photometric prompt learning and consensus attention regularization.

Background & Motivation

Face anti-spoofing (FAS) serves as an essential defense layer in facial biometric authentication systems against diverse presentation attacks such as printed photographs, video replay screens, and sophisticated 3D silicone masks. While modern deep neural networks and vision-language foundation models achieve near-saturated accuracy in intra-dataset benchmarks, their verification performance deteriorates sharply when evaluated on unseen target domains. This generalization bottleneck stems largely from the fact that FAS classifiers are prone to overfitting to domain-specific capture conditions, sensor noise, illumination profiles, and color distributions, erroneously treating these non-liveness environmental discrepancies as spoofing artifacts.

To mitigate domain shift, conventional unsupervised domain adaptation (UDA) and test-time adaptation (TTA) approaches align source and target feature distributions via iterative gradient updates. However, in practical deployment scenarios, gathering diverse spoofing samples from unseen deployment domains is prohibitively labor-intensive and practically infeasible. Furthermore, performing test-time back-propagation or generative style transfer on edge devices incurs severe computational latency, memory footprints, and potential catastrophic model collapse. Conversely, acquiring merely a few genuine, bonafide facial images within a new operating environment is remarkably inexpensive and convenient.

This contrast presents a pivotal challenge: how can a model achieve generalized FAS adaptation using only a handful of target-domain bonafide samples without updating network weights? The authors recognize that all faces captured within a shared environment inherently inherit identical capture-device characteristics and ambient lighting profiles. Core idea: by utilizing a few target-domain bonafide faces as reference anchors, TRAM frames FAS as an anchor-relative image-pair-text contrastive learning task, removing shared environmental biases via feature subtraction and mining fine-grained photometric discrepancy prompts to achieve fine-tuning-free test-time adaptation.

Method

Overall Architecture

TRAM takes a pair of images as input—the incoming query face and a bonafide anchor image from the target domain—and determines whether the query is live or spoofed by assessing their relative difference. The system encodes image features using a pre-trained, frozen CLIP visual backbone and maps them through an anchor-relative FAS encoder. The resulting relational representation is matched against relational text descriptions in the text embedding space. To capture subtle presentation medium flaws, a fine-grained relation prompt learning module extracts low-level photometric differences and dynamically appends descriptive artifact prompts. In addition, consensus attention regularization is imposed to prevent the model from overfitting to isolated spoofing cues.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Input Query Ix & Target Bonafide Anchor IA"] --> B["Anchor-Relative Cross Attention RCA<br/>Q/K feature subtraction with facial mask modulation"]
    A --> C["Fine-Grained Relation Prompt Learning FRPL<br/>ACA mines photometric differences like contrast & white balance"]
    C --> D["Conditional Attribute-Balanced Loss Lfg<br/>Dynamically generates fine-grained spoof prompts t+D"]
    B --> E["Relative Image-Pair-Text Contrastive Learning I2CL<br/>Matches relational visual features with adaptive prompts"]
    D --> E
    E --> F["Consensus Attention Regularization CAR<br/>Maintains category Grad-CAMs via EMA & minimizes JS divergence"]
    F --> G["Fine-Tuning-Free Live / Spoof Decision"]

Key Designs

1. Anchor-Relative Cross Attention (RCA): Suppressing Shared Domain Factors via Feature Subtraction

Standard vision-language FAS models typically match individual facial images directly to prompts such as "This is a real/spoof face", which inadvertently encodes domain-specific background noise and camera sensor characteristics. TRAM builds an anchor-relative FAS encoder \(f_R\), whose initial layer is an Anchor-Relative Cross Attention (RCA) mechanism. For visual features \(x\) (query) and \(A\) (bonafide anchor), linear projections yield query, key, and value matrices (\(Q_x, K_x, V_x\) and \(Q_A, K_A, V_A\)). To cancel out non-FAS commonalities shared between the two images within the same environment, RCA explicitly computes difference representations across queries and keys: $\(\mathrm{RCA}(Q_A, Q_x, K_A, K_x, V_A) = \mathrm{Softmax}\left( \frac{(Q_x - Q_A)(K_x - K_A)^\top}{\sqrt{d} \cdot \mathcal{T}(M_x, M_A)} \right) V_A\)$ where \(M_x\) and \(M_A\) denote facial prior masks and \(\mathcal{T}(\cdot)\) is a temperature modulation function that emphasizes informative facial regions. By subtracting the anchor representation, shared facial identity factors and background illumination biases are neutralized, directing attention toward anomalous spoofing discrepancies.

2. Fine-Grained Relation Prompt Learning (FRPL): Mining Low-Level Photometric Discrepancies

Presentation attack instruments (such as paper grain, digital screen pixels, or silicone surfaces) naturally introduce physical distortions in low-level photometric properties including contrast, color saturation, colorfulness, and white balance. Since existing FAS datasets lack fine-grained attribute annotations, TRAM introduces an anchorized spoof artifact extractor \(f_A\) based on Anchorize Cross Attention (\(\mathrm{ACA}(Q_x, K_A, V_A)\)). It calculates ratio differences of pre-computed photometric statistics between pairs on the fly to establish pseudo-labels. To tackle label noise and class imbalance, a conditional attribute-balanced cross-entropy loss is formulated: $\(\mathcal{L}_{fg} = -\frac{1}{D}\sum_{d=1}^D \left[ \alpha \cdot \mathbb{I}(y = \text{spoof}) \log \hat{y}^+_d + (1 - \alpha \cdot \mathbb{I}(y = \text{spoof})) \log \hat{y}^-_d \right]\)$ where \(D\) represents the number of photometric attributes and \(\alpha\) adjusts category weighting. During inference, attributes identified as anomalous are dynamically concatenated (\(\cup\)) into the spoof class prompt text \(t_s \leftarrow t_s \cup t^+_D\), yielding a highly discriminative relational textual description.

3. Consensus Attention Regularization (CAR): Preventing Overfitting to Local Shortcuts

Due to the intrinsic attention sparsity of vision-language architectures, models trained with few target examples risk latching onto superficial domain-specific shortcuts (e.g., finger borders holding a print attack or eye cutouts on masks). TRAM addresses this by maintaining class-specific consensus attention maps \(\bar{G}_c\) for genuine samples and distinct spoof types. Throughout training iterations, the sample-level Grad-CAM map \(G_c^{(i)}\) is tracked and integrated via exponential moving average: $\(\bar{G}_c^{(n+1)} = \mu \bar{G}_c^{(n)} + (1 - \mu) G_c^{(i)}\)$ Once the consensus distributions stabilize, a consensus attention regularization loss \(\mathcal{L}_{car}\) minimizes the Jensen-Shannon divergence (\(\mathrm{JS}(G_c^{(i)} \parallel \bar{G}_c)\)) between individual sample attention patterns and their corresponding class consensus, driving the model to preserve coherent, topologically generalized attention over spoofing cues.

Loss & Training

To avoid optimization instability caused by the mutual dependency between the relational encoder \(f_R\) and the artifact extractor \(f_A\), TRAM adopts a sequential three-stage training schedule: 1. Stage 1 (Basic Relational Modeling): The visual encoder and \(f_R\) are optimized with standard classification loss \(\mathcal{L}_{cls}\) using basic relation prompts ("This is a {real/spoof} face. The face is {the same as/different from} the anchor.") to establish basic discrimination capability. 2. Stage 2 (Photometric Artifact Extraction): Freezing the visual backbone and \(f_R\), the artifact extractor \(f_A\) is trained under the conditional attribute-balanced loss \(\mathcal{L}_{fg}\) using pseudo-labels generated from photometric ratio metrics. 3. Stage 3 (Joint Optimization): The relational encoder \(f_R\) is fine-tuned under the composite objective \(\mathcal{L}_{sum} = \mathcal{L}_{cls} + \lambda_1 \mathcal{L}_{car} + \lambda_2 \mathcal{L}_{fg}\), integrating fine-grained artifact prompt guidance. Hyperparameters are configured as \(\alpha=0.9, \theta=0.9, \mu=0.9, \lambda_1=1, \lambda_2=1\). During test-time deployment, five bonafide target images serve as candidate anchors, and the anchor closest in overall illumination intensity is selected for pairing without updating any model parameters.

Key Experimental Results

Main Results

TRAM was benchmarked across eight widely recognized public FAS datasets under five evaluation protocols, measuring Half Total Error Rate (HTER, lower is better), Area Under the ROC Curve (AUC, higher is better), and TPR@FPR=1%.

Leave-One-Out Cross-Dataset Evaluation under Protocol 2 (HTER / %):

Source → Target	Metric	TRAM (Ours)	SOTA (BUDoPT)	Gain over SOTA
O&C&I → M	HTER	0.00	0.40	-0.40%
O&M&I → C	HTER	0.11	0.26	-0.15%
O&C&M → I	HTER	1.85	1.38	+0.47%
I&C&M → O	HTER	1.27	1.60	-0.33%
Average	HTER	0.81	0.91	-0.10%

In Protocol 1 (one-to-one cross-dataset evaluation across M, C, I, and O), TRAM achieved an average HTER of 1.65%, outperforming the vision-language baseline TF-FAS (1.99%) and traditional test-time adaptation methods like GDA (14.4%). Under the challenging multi-ethnic, multi-modal Protocol 3 (WMCA, CASIA-CeFA, CASIA-SURF), TRAM attained an average HTER of 8.45% (compared to MVP-FAS at 12.92%). In Protocol 4 evaluating high-quality 3D silicone mask attacks (WMCA → HKBUMARsV2+), TRAM recorded an HTER of 0.91% and AUC of 99.83%, outperforming the FLIP family baselines (best at 1.97% HTER).

Ablation Study

Ablation experiments conducted under Protocol 2 validate the individual impact of each architectural module (HTER / %):

Module-Level Ablation under Protocol 2:

Config	I2CL	CAR	FRPL	OCI→M	OMI→C	OCM→I	ICM→O	Avg. HTER	Note
Baseline (FLIP-IT)	-	-	-	5.27	0.44	2.94	2.31	3.05	Single-image vision-language alignment
+ I2CL	✓	-	-	0.24	0.67	4.05	1.87	1.70	Anchor-relative modeling (-44% HTER)
+ I2CL + CAR	✓	✓	-	0.00	0.33	3.55	1.41	1.32	Regularized attention consensus
Full Model	✓	✓	✓	0.00	0.11	1.85	1.27	0.81	Integrated photometric prompt guidance

RCA Operator-Level Analysis (Protocol 2 Average HTER): - Siamese dual-branch concatenation: 3.66% - Standard cross-attention baseline: 2.80% - RCA with Query-difference only: 2.40% - RCA with Key-difference only: 2.46% - RCA with Query and Key difference (without mask modulation): 2.04% - Full RCA (Q/K subtraction + facial prior mask temperature modulation): 1.70%

Key Findings

• Sample Efficiency with Minimal Anchors: The adaptation capability stabilizes with as few as five bonafide target samples. Increasing the number of shots beyond five yields negligible variance in AUC, ensuring minimal cold-start data acquisition costs in practice.
• Robustness Across Anchor Selection Heuristics: Comparing selection criteria based on deep semantic embeddings (ResNet-18), color histograms, and global image intensity, simple intensity-based matching achieves the lowest average HTER (0.81% vs. semantic 0.97%). High-level semantics appear to inject identity-correlated biases, whereas intensity alignment directly mitigates environmental lighting variance.
• One-Class Attention Clustering Behavior: Grad-CAM inspection reveals that bonafide test inputs yield tightly focused, spatially coherent facial attention that mirrors the anchor's attention layout. In contrast, spoof samples trigger scattered, inconsistent attention distributions across peripheral areas.

Highlights & Insights

• Reformulating Domain Adaptation as Pairwise Relational Modeling: Rather than enforcing distribution alignment through adversarial training or source-to-target style translation, TRAM exploits the physical insight that all images in a target domain share the same capture conditions, converting test-time adaptation into an anchor-relative difference classification task.
• Physics-Grounded Photometric Discrepancy Prompting: Instead of relying solely on general LLM-generated semantic descriptions, TRAM computes photometric difference statistics (contrast, saturation, colorfulness, white balance) to synthesize fine-grained attribute prompts via a conditioned attribute-balanced loss.
• Truly Fine-Tuning-Free Deployment: All parameters remain frozen during target-domain inference, completely eliminating test-time back-propagation, mitigating memory constraints, and guarding against catastrophic model degradation.

Limitations & Future Work

• Sensitivity to Extreme Lighting Discrepancies: If an incoming face is captured under severe directional backlight while the candidate bonafide anchors are evenly illuminated, heuristic intensity matching might pair incompatible references, causing false rejections of real faces.
• Absence of Direct Spoof Guidance for Zero-Day Media: Since no attack samples are available from the target domain, the photometric prompts rely on source-domain heuristics, which might fail to capture novel physical artifacts from unseen display or printing hardware.
• Future Directions: Incorporating head pose and 3D facial geometry priors into anchor selection could enhance robustness under diverse camera angles and irregular shadows.

Related Work & Insights

• vs. FLIP / BUDoPT / TF-FAS: While vision-language FAS models align individual images with general text prompts, they remain vulnerable to domain shifts in low-level image attributes. TRAM conditions classification on anchor-relative differences, effectively filtering out shared environmental biases.
• vs. ViTAF / SDA-FAS++: Prior test-time adaptation methods update model parameters using target-domain batches or few-shot samples, requiring both compute resources and carefully calibrated balance between real and fake samples. TRAM achieves superior generalization in a fully feed-forward, parameter-frozen regime.

Rating

• Novelty: ⭐⭐⭐⭐⭐ Elegant reformulation of test-time FAS adaptation via anchor-relative modeling and photometric prompt mining.
• Experimental Thoroughness: ⭐⭐⭐⭐⭐ Comprehensive cross-domain protocols across eight datasets, supplemented by operator-level ablations.
• Writing Quality: ⭐⭐⭐⭐⭐ Well-structured narrative with rigorous mathematical formulation and lucid figure explanations.
• Value: ⭐⭐⭐⭐⭐ Significant practical utility for low-cost, plug-and-play biometric authentication across edge devices.