CausalCLIP: Causally-Informed Feature Disentanglement and Filtering for Generalizable Detection of Generated Images

Conference: AAAI 2026 arXiv: 2512.13285
Code: Unavailable
Area: Image Generation Keywords: Generated image detection, causal reasoning, CLIP, feature disentanglement, adversarial training

TL;DR

CausalCLIP is proposed to disentangle CLIP features into causal and non-causal subspaces via Gumbel-Softmax masking and HSIC constraints, combined with adversarial masking and counterfactual intervention to preserve stable forensic cues, achieving a 6.83% accuracy improvement in cross-generator generalization.

Background & Motivation

The rapid advancement of generative models (GANs, diffusion models) has substantially lowered the barrier to producing high-quality forged images, creating an urgent need for detection methods that generalize across diverse generators. Existing approaches face the following challenges:

Generator-specific detection: Early CNN-based classifiers rely on generator-specific artifacts such as upsampling traces and frequency-domain anomalies, resulting in poor generalization to unseen generators.

Entangled CLIP feature space: Recent methods leveraging pretrained vision-language models such as CLIP (UnivFD, CLIPping, C2P-CLIP) show improvement but still operate in highly entangled feature spaces where causal forensic cues are mixed with spurious patterns.

Coarse-grained filtering: VIB-Net suppresses irrelevant features via an information bottleneck but does not explicitly disentangle causal from non-causal components, resulting in coarse filtering that may discard task-relevant features.

Root cause: Achieving cross-generator generalization requires first explicitly separating causal forensic cues from spurious artifacts, and then performing filtering in the disentangled space (disentangle-then-filter), rather than applying coarse suppression in an entangled space.

Method

Overall Architecture

CausalCLIP follows the disentangle-then-filter paradigm:

1. A frozen CLIP-ViT-L/14 encoder extracts image features.
2. Factorization Module: Disentangles CLIP features into causal and non-causal components.
3. Adversarial Masking Module: Suppresses the influence of non-causal features via an adversarial mechanism.
4. The disentangled causal features are fed into a lightweight classifier for real/fake prediction.

Key Designs

1. Factorization Module

It is assumed that an image \(X\) is generated by a structural causal model containing two independent factors: \(Z_c\) (causal features, derived from a generator-agnostic content factor G) and \(Z_{nc}\) (non-causal features, derived from a generator-specific style/artifact factor C).

Given the CLIP embedding \(E = \text{CLIP}(X) \in \mathbb{R}^d\), a feature mask \(M \in [0,1]^d\) is learned to perform separation:

\[\tilde{Z}_c = M \odot E, \quad \tilde{Z}_{nc} = (1-M) \odot E\]

The mask \(M\) is parameterized via Gumbel-Softmax:

\[M = \sigma((\text{MLP}(E) + g) / \tau), \quad g \sim \text{Gumbel}(0,1)\]

The temperature \(\tau\) controls the sparsity of feature selection. This differentiable feature selection mechanism ensures the purity of the causal subspace \(\tilde{Z}_c\).

2. Adversarial Masking Module

Even after factorization, residual non-causal signals may still influence the classifier. A minimax game is therefore introduced:

• Classifier h: Predicts real/fake based on causal features \(\tilde{Z}_c\).
• Adversary d: Attempts to predict real/fake from non-causal features \(\tilde{Z}_{nc}\).

The classifier and mask are optimized to ensure \(\tilde{Z}_{nc}\) contains no discriminative information, forcing the model to rely solely on \(\tilde{Z}_c\).

Mask regularization:

\[\mathcal{L}_{\text{mask}} = \lambda_1 \|M\|_1 + \lambda_2 \widehat{\text{HSIC}}(\tilde{Z}_c, \tilde{Z}_{nc})\]

• The \(\ell_1\) norm encourages sparse feature selection.
• The empirical HSIC term encourages statistical independence between the causal and non-causal subspaces (Gaussian kernel with median heuristic bandwidth selection).

3. Counterfactual Intervention

Random masking is applied to causal features to simulate distributional perturbations:

\[\tilde{Z}_c^{CF} = \tilde{Z}_c \odot (1 - B), \quad B \sim \text{Bernoulli}(p)\]

Prediction consistency is enforced via KL divergence:

\[\mathcal{L}_{\text{inv}} = \text{KL}(h(\tilde{Z}_c) \| h(\tilde{Z}_c^{CF}))\]

This compels the classifier to rely on stable causal semantics rather than generator-dependent cues.

Loss & Training

The overall objective is:

\[\mathcal{L}_{\text{total}} = \mathcal{L}_{\text{cls}} - \alpha \mathcal{L}_{\text{adv}} + \mathcal{L}_{\text{mask}} + \beta \mathcal{L}_{\text{inv}}\]

Training alternates among: minimizing \(\mathcal{L}_{\text{cls}} + \beta \mathcal{L}_{\text{inv}}\) for h, maximizing \(\mathcal{L}_{\text{adv}}\) for d, and minimizing \(\mathcal{L}_{\text{total}}\) for M.

Implementation details: Adam optimizer, lr=1e-4, batch=256, center-cropped input at 224×224, trained on a V100 GPU.

Key Experimental Results

Main Results

Table 1: AP on 15 test sets, trained on Diffusion source (SDv1.4)

Method	SD1.4	ADM	GLIDE	ProGAN	CycleGAN	StyleGAN	GauGAN	Mean AP
CNNSpot	99.98	51.10	58.80	53.15	50.23	49.79	47.07	63.24
UnivFD	96.04	66.34	93.73	51.77	63.42	75.81	54.12	75.31
VIB-Net	100.00	95.49	97.13	96.59	98.44	97.17	84.31	94.60
CausalCLIP	100.00	95.37	99.47	97.33	98.35	96.29	98.62	96.92

Table 2: ACC on 15 test sets, trained on Diffusion source

Method	Mean ACC
CLIPping	83.03
VIB-Net	85.83
CausalCLIP	90.45

CausalCLIP achieves +2.32% mean AP and +4.62% ACC over VIB-Net. On unseen GAN-based generators, it yields an additional +6.83% ACC and +4.06% AP improvement.

Tables 3–4: Cross-diffusion generalization, trained on GAN source (ProGAN)

Method	Mean AP	Mean ACC
VIB-Net	93.04	82.97
CausalCLIP	94.27	86.23

CausalCLIP maintains superiority under the more challenging GAN→Diffusion distribution shift.

Ablation Study

Table 5: Module contribution ablation (trained on SDv1.4)

Factorization	Masking	ACC(OP)	ACC(GP)	AP(OP)	AP(GP)
✗	✗	65.37	60.42	75.31	67.09
✓	✗	79.42	75.91	89.53	87.78
✗	✓	70.73	65.94	82.13	79.28
✓	✓	90.45	89.95	96.92	95.25

The factorization module alone yields +14.05% ACC; the masking module alone yields +5.36%; their combination achieves the best overall performance.

Key Findings

1. UMAP visualization: Raw CLIP features are highly entangled between real and fake images; VIB achieves only partial separation, while CausalCLIP achieves clear separation on both seen and unseen generators.
2. Robustness: Under JPEG compression and Gaussian blur perturbations, CausalCLIP maintains the most stable performance.
3. The improvement from the factorization module substantially exceeds that of the masking module, confirming that explicit disentanglement is the critical factor.

Highlights & Insights

• Theory-driven design: The relationship between causal and non-causal features is formalized using a structural causal model, rather than treated as simple feature selection.
• Gumbel-Softmax + HSIC combination: The former ensures differentiable sparse selection; the latter ensures statistical independence between causal and non-causal subspaces, grounding the method in sound theoretical principles.
• Lightweight: Only downstream modules are trained on top of frozen CLIP, with computational overhead comparable to UnivFD but substantially improved performance.

Limitations & Future Work

1. The method relies on fixed features from CLIP-ViT-L/14 and may be sensitive to artifact types not well represented in CLIP's pretraining data.
2. The binary classification setting (real/fake) does not support generator attribution.
3. The mask \(M\) operates as an element-wise mask over the entire embedding dimension; finer granularity (e.g., patch-level disentanglement) may be preferable.
4. The selection of the Bernoulli probability \(p\) for counterfactual intervention lacks theoretical justification.

Related Work & Insights

• Compared to VIB-Net's information bottleneck approach, CausalCLIP's advantage lies in explicit disentanglement prior to filtering, avoiding the loss of useful features that can occur in entangled spaces.
• The successful application of causal representation learning (CausalVAE, DEAR) to image forensics provides a reference for other distribution-shift scenarios.
• The disentangle-then-filter paradigm is extensible to deepfake video detection, machine-generated text detection, and related domains.

Rating

• Novelty: ★★★★☆ — First systematic application of causal disentanglement + adversarial masking to generated image detection
• Technical Depth: ★★★★☆ — Theoretically complete, integrating SCM modeling, HSIC, Gumbel-Softmax, and counterfactual intervention
• Experimental Thoroughness: ★★★★☆ — Comprehensive evaluation across 15 test sets, two training sources, with ablation, robustness, and visualization studies
• Writing Quality: ★★★★☆ — Motivation is clearly articulated; methodological presentation is rigorous
• Value: ★★★★☆ — Lightweight training on frozen CLIP enables straightforward deployment

Related Papers

• [CVPR 2026] Diversity over Uniformity: Rethinking Representation in Generated Image Detection
• [AAAI 2026] Aggregating Diverse Cue Experts for AI-Generated Image Detection
• [ECCV 2024] Zero-Shot Detection of AI-Generated Images
• [AAAI 2026] Beautiful Images, Toxic Words: Understanding and Addressing Offensive Text in Generated Images
• [AAAI 2026] Beyond Semantic Features: Pixel-Level Mapping for Generalized AI-Generated Image Detection