Prior-Conditioned Gaussian Discriminants for Generalizable AI-generated Image Detection

Conference: ECCV 2026
Paper: ECCV 2026
Area: AI Safety / AIGC Detection
Keywords: AI-generated image detection, prior-conditioned Gaussian discriminants, covariance ladder, distribution shift, transfer learning

TL;DR

Formalizes AI-generated image (AIGI) detection as a tripartite transfer system of training prior, frozen representation, and decision rule, establishing a closed-form Gaussian discriminant ladder with nested covariance assumptions as a diagnostic baseline that reveals learned classifier heads often fail to surpass frozen low-order feature geometry.

Background & Motivation

The ubiquitous spread of diffusion models and cutting-edge generative architectures has enabled the creation of highly realistic synthetic imagery, creating urgent societal challenges for visual authentication, copyright tracing, and malicious misinformation mitigation. Current detection models frequently achieve near-saturated accuracy under in-distribution (IID) test environments. However, once deployed into real-world scenarios, detectors suffer catastrophic performance degradation under simultaneous shifts in generative model families (e.g., transitions from GANs to diverse latent diffusion architectures), prompt/artistic styles, and acquisition sources along with image post-processing pipelines. To counter such distribution shift, recent research has pursued two major trajectories: expanding large-scale training corpora across multiple generators, or attaching learned parameter-heavy classifier heads (e.g., MLPs or LoRA adapters) on top of foundation vision model representations.

Nevertheless, evaluating detectors as monolithic end-to-end entities introduces severe attribution ambiguity. When an AIGI detection system fails on an unseen generator or domain, it remains ambiguous whether the bottleneck stems from inadequate representation capacity in the frozen visual encoder, overfitting to generator-specific shortcuts in the training prior, or destructive distortions induced during classifier head gradient descent. Furthermore, existing evaluation benchmarks predominantly alter either the generator family or the image source domain in isolation, failing to simulate joint real-world distribution drift. Crucially, many publications claim that sophisticated learned heads yield out-of-distribution transfer gains, yet lack controlled comparisons against classical closed-form statistical decision rules fitted under identically matched priors and frozen features.

The key insight of this paper is to explicitly decouple AIGI detection into three orthogonal dimensions: the training prior distribution, the frozen image encoder, and the decision rule operating on feature space. The authors advocate that prior to pursuing complicated neural classifier heads, the community should establish an optimization-free diagnostic baseline constructed purely from first- and second-order feature statistics to probe whether head training contributes genuine value beyond low-order geometry. Core idea: model AIGI detection as a prior-conditioned feature transfer diagnostic system and establish a closed-form Gaussian discriminant ladder with nested covariance assumptions (isotropic, diagonal, shared full, and class-specific full) to audit existing detection heads and delimit generalization bounds under matched prior and representation conditions.

Method

Overall Architecture

The proposed framework formalizes AIGI detection as support-conditioned binary inference over a frozen embedding space. Given an input query image \(x\), a pre-trained frozen foundation image encoder \(\varphi\) maps it to a \(D\)-dimensional feature embedding \(z = \varphi(x) \in \mathbb{R}^D\). During training or calibration, rather than running iterative gradient descent on classifier weights, the method draws real and synthetic support samples from a specified public training prior distribution \(\mathcal{D}_{\text{train}}\) to estimate class-conditional sample mean vectors \(\mu_c\) and covariance matrices \(\Sigma_c\) (\(c \in \{0, 1\}\) for real and fake). By imposing nested structural assumptions on the covariance model, the framework instantiates a four-rung Gaussian discriminant ladder. During inference, arbitrary unseen test embeddings are directly scored and classified via closed-form analytic expressions.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Input: Multi-Source Images<br/>x ∈ D_train (Real and Synthetic)"] --> B["Frozen Vision Foundation Encoder<br/>z = φ(x) high-dimensional embedding"]
    B --> C["Prior-Conditioned Moment Estimation & Regularization<br/>Compute (μ_c, Σ_c) and Σ_p on support set"]
    C --> D["Nested Covariance Discriminant Ladder<br/>Analytical Euc/Cos-NCM, GNB, Mah-NCM, QDA"]
    D --> E["Matched-Condition Audit & Diagnostic Upper Bound<br/>Contrast learned heads vs. ladder under matched prior & encoder"]
    E --> F["Output: OOD Generalization Diagnostics & Binary Decision"]

Key Designs

1. Prior-Conditioned Moment Estimation & Regularization: Extracting first- and second-order statistics without gradient optimization

To eliminate optimization artifacts and gradient-induced feature distortions, this design estimates empirical class-conditional moments directly within the frozen feature manifold. For a given training support prior \(\mathcal{S} \subseteq \mathcal{D}_{\text{train}}\), empirical mean vectors and covariance matrices for real (\(c=0\)) and fake (\(c=1\)) classes are obtained via standard sample expectation. Under the homoscedastic assumption, the pooled covariance matrix is computed as:

\[\Sigma_p = \frac{(n_0 - 1)\Sigma_0 + (n_1 - 1)\Sigma_1}{n_0 + n_1 - 2}\]

In high-dimensional embedding regimes (e.g., ViT representations where \(D \ge 768\)), empirical covariance inversion and log-determinant operations suffer from ill-conditioning or near-singularity. To resolve this numerical instability, standard covariance shrinkage and diagonal loading regularizations are integrated: \(\hat{\Sigma} = (1 - \alpha)\Sigma + \alpha \frac{\text{Tr}(\Sigma)}{D}I + \lambda I\). This formulation guarantees stable inversion of \(\Sigma^{-1}\) even when calibrated on minimal support sets (e.g., several dozen images), reliably preserving the dominant variance orientations of the feature space.

2. Nested Covariance Discriminant Ladder: Closed-form geometric boundaries from spherical to heteroscedastic forms

To identify which level of statistical granularity effectively transfers across domain and generator shifts, the paper introduces a bottom-up hierarchical ladder of four classical Gaussian decision rules: - Spherical Covariance Rung (Euc-NCM / Cos-NCM): Assumes an identical isotropic covariance \(\Sigma_c = \sigma^2 I\) across both classes. The log-likelihood ratio simplifies to Euclidean nearest-centroid distance comparisons; under \(\ell_2\)-normalized features, this reduces to Cosine Nearest Centroid Matching (Cos-NCM), producing a linear separating hyperplane orthogonal to the centroid difference vector. - Diagonal Covariance Rung (GNB): Imposes class-conditional feature independence, setting \(\Sigma_c = \text{diag}(\sigma_{c,1}^2, \dots, \sigma_{c,D}^2)\). This model accounts for per-dimension feature variance while disregarding cross-feature correlations, generating axis-aligned quadratic boundaries. - Homoscedastic Full Covariance Rung (Mah-NCM): Assumes real and fake distributions share an identical elliptical covariance geometry (\(\Sigma_0 = \Sigma_1 = \Sigma_p\)). Decision scoring relies on the Mahalanobis distance \((z - \mu_c)^\top \Sigma_p^{-1} (z - \mu_c)\), which whitens the embedding space to eliminate global feature correlation, yielding a correlation-aware Bayes-optimal linear discriminant analysis (LDA) boundary. - Heteroscedastic Full Covariance Rung (QDA): Grants each class an independent, unconstrained full-rank covariance matrix (\(\Sigma_0 \neq \Sigma_1\)), inducing a flexible quadratic decision surface:

\[\delta_c(z) = -\frac{1}{2}\ln|\Sigma_c| - \frac{1}{2}(z - \mu_c)^\top \Sigma_c^{-1}(z - \mu_c) + \ln P(y=c)\]

Because these four formulations represent strictly nested geometric constraints, the optimal performing rung directly pinpoints which statistical moments successfully preserve transferable signals across shifts.

3. Matched-Condition Audit & Diagnostic Upper Bound: Disentangling encoder, prior, and classifier head contributions

Departing from the conventional practice of treating detectors as monolithic black-boxes, this design introduces a controlled three-factor audit protocol spanning (training prior, frozen encoder, classifier head). For any existing detection model releasing internal feature representations, researchers extract embeddings on the exact training prior used by that system and fit the closed-form Gaussian ladder. If an iteratively trained classification head fails to outperform the best closed-form Gaussian rung (typically Mah-NCM) under matched prior and representation conditions, it demonstrates that reported OOD generalization gains originate from intrinsic feature separability rather than the optimization objective of the learned head. Conversely, when a trained head substantially outperforms the ladder, it provides concrete empirical verification that the data contains non-Gaussian higher-order structure that linear and quadratic second-order baselines cannot capture.

A Worked Example

The complete inference procedure for Mah-NCM using a frozen foundation encoder unfolds as follows: 1. Calibration via Support Set: From the CommunityForensics prior, draw a tiny support subset (e.g., 0.005%, corresponding to 219 images) and feed them into a frozen PE-Core-bigG-14-448 backbone to generate 1536-dimensional embeddings. Compute class mean vectors \(\mu_0, \mu_1\) and the regularized pooled inverse covariance \(\Sigma_p^{-1}\). 2. Feature Extraction: Feed an unseen test query \(x_{\text{test}}\) through the frozen visual encoder to extract representation \(z \in \mathbb{R}^{1536}\). 3. Closed-form Distance Scoring: Compute the squared Mahalanobis distances to the real centroid \(d_M(z, \mu_0)^2 = (z - \mu_0)^\top \Sigma_p^{-1} (z - \mu_0)\) and fake centroid \(d_M(z, \mu_1)^2 = (z - \mu_1)^\top \Sigma_p^{-1} (z - \mu_1)\). 4. Classification: Assign the synthetic label if \(d_M(z, \mu_1)^2 < d_M(z, \mu_0)^2\). This process executes in milliseconds without back-propagation, learning rates, or early-stopping heuristics, reaching a macro-average accuracy of 90.89% across the Percept-Lens benchmark.

Loss & Training

Because the Gaussian discriminant ladder utilizes closed-form analytic solutions, it requires no gradient-based loss optimization. Theoretical guarantees from Bayesian decision theory provide the underlying foundation: - Bayes Optimality: Under exact Gaussian class-conditional distributions and equal class priors, QDA constitutes the Bayes-optimal classifier; under the additional homoscedasticity assumption, Mah-NCM provides the Bayes-optimal linear decision surface. - Performance Stability under Bounded Drift: If statistical drift between test and train distributions satisfies \(\|\mu_c^{\text{test}} - \mu_c^{\text{train}}\|_2 \le \epsilon_\mu\) and \(\|\Sigma^{\text{test}} - \Sigma^{\text{train}}\|_F \le \epsilon_\Sigma\), the AUC difference under homoscedasticity is strictly bounded by Lipschitz continuity: \(|\text{AUC}_{\text{test}} - \text{AUC}_{\text{train}}| \le L(\epsilon_\mu + \epsilon_\Sigma)\). - Fisher Margin Stability: For mean difference vector \(\delta = \mu_1 - \mu_0\), distributional drift perturbs the Fisher margin \(\mathcal{D}_\mu\) smoothly according to \(|\mathcal{D}_\mu^{\text{test}} - \mathcal{D}_\mu^{\text{train}}| \le 2 \|(\Sigma_p^{\text{train}})^{-1}\|_2 \|\delta^{\text{train}}\|_2 \epsilon + O(\eta)\), demonstrating graceful performance degradation as long as low-order geometric alignment holds.

Key Experimental Results

Main Results

To stress-test detectors under concurrent shifts in generator families, prompt styles, and source domains, experiments were conducted on the unified Percept-Lens suite encompassing 39 public datasets and 7.1 million images. The table below compares out-of-the-shelf released detector heads against the best closed-form Gaussian ladder rung under strictly matched training priors and frozen encoder representations (macro-averaged balanced Class Accuracy CA):

Detection Model	Backbone / Feature Type	Training Prior Dataset	Out-of-the-Shelf (CA)	Euc-NCM (CA)	Best Gaussian Rung (CA)	Gain
UnivFD	ViT-L-14 (OpenAI)	CNNSpot (LSUN ProGAN)	54.63%	50.98%	57.62% (Mah-NCM)	+2.99%
AIDE	Multi-expert frequency + semantic	CNNSpot (LSUN ProGAN)	56.41%	54.25%	62.76% (Mah-NCM)	+6.35%
AIDE	Multi-expert frequency + semantic	GenImage-SDv1 (ImageNet-1k)	48.99%	54.53%	57.35% (Mah-NCM)	+8.36%
Effort	LoRA adapted features	GenImage-SDv1 (ImageNet-1k)	72.58%	75.06%	76.82% (Mah-NCM)	+4.24%
DRCT-UnivFD	Reconstruction contrastive features	Full GenImage	65.77%	66.08%	73.01% (Mah-NCM)	+7.24%
AIDE	Multi-expert frequency + semantic	Full GenImage	54.98%	52.24%	64.64% (Mah-NCM)	+9.66%
DRCT-UnivFD	Reconstruction contrastive features	DRCT-SDv1 (COCO)	63.36%	65.36%	70.20% (QDA)	+6.84%
DRCT-UnivFD	Reconstruction contrastive features	DRCT-SDv2 (COCO)	62.37%	66.31%	70.31% (QDA)	+7.94%
CoDE-kNN	End-to-end contrastive features	ELSA-D3 (LAION-400M)	64.01%	63.63%	66.15% (Mah-NCM)	+2.14%
CF-224	End-to-end fine-tuned backbone	CommunityForensics	81.98%	78.53%	82.61% (Mah-NCM)	+0.63%
CF-384	End-to-end fine-tuned backbone	CommunityForensics	87.54%	82.65%	84.45% (Mah-NCM)	-2.99%
Frozen Foundation	PE-Core-bigG-14-448	No detection fine-tuning	—	86.38%	94.46% (Mah-NCM)	—

Ablation Study

The table below illustrates performance across the Gaussian discriminant ladder when fixing the frozen backbone to PE-Core-bigG-14-448 and varying only the support prior used to estimate moments:

Training Support Prior	Euc-NCM (Spherical)	Cos-NCM (Cosine)	GNB (Diagonal)	Mah-NCM (Homoscedastic)	QDA (Heteroscedastic)	Winning Covariance
CNNSpot (ProGAN)	74.95%	78.57%	59.21%	77.93%	49.24%	Cos / Mah
GenImage (Multi-Diffusion)	86.68%	88.75%	87.50%	92.43%	81.83%	Mah-NCM
GenImage-SDv1	88.74%	89.80%	86.34%	92.57%	65.06%	Mah-NCM
DRCT-2M (SD-COCO)	81.90%	81.76%	86.21%	83.25%	91.09%	QDA
DRCT-SDv1	82.29%	83.81%	88.70%	88.31%	89.92%	QDA
DRCT-SDv2	83.50%	84.38%	89.56%	90.08%	91.44%	QDA
ELSA-D3 (Multi-Diffusion)	88.87%	89.03%	86.98%	94.45%	86.40%	Mah-NCM
CommunityForensics	86.38%	86.06%	87.26%	94.46%	86.15%	Mah-NCM

To evaluate sample efficiency, support set subsampling was evaluated using CommunityForensics moments on top of PE-Core-bigG-14-448:

Support Sample Ratio	Approximate Sample Count	Euc-NCM (CA)	Cos-NCM (CA)	GNB (CA)	Mah-NCM (CA)	QDA (CA)
0.001%	~44 samples	83.97%	83.46%	87.14%	85.56%	85.13%
0.005%	~219 samples	86.04%	85.78%	87.23%	90.89%	85.53%
0.01%	~438 samples	86.32%	86.02%	87.42%	91.55%	86.48%
0.1%	~4,380 samples	86.26%	85.92%	87.33%	93.18%	84.18%
1.0%	~43,800 samples	86.39%	86.05%	87.28%	93.52%	85.16%
100%	Full multi-million set	86.38%	86.06%	87.26%	94.46%	86.15%

Key Findings

• Learned heads frequently fail to surpass second-order baselines: Across 11 matched-condition detector evaluations, 10 released heads were surpassed by simple closed-form Gaussian rungs (with margins up to +9.66% on AIDE-Full GenImage). The primary exception is CF-384 (87.54% vs 84.45%), indicating that specialized head training predominantly captures spurious training-set artifacts rather than generalizable OOD discriminants.
• Training prior dominates out-of-distribution transfer: Holding the feature extractor fixed (PE-Core-bigG), modifying only the calibration prior alters CA performance by 16.53 percentage points (77.93% on CNNSpot vs. 94.46% on CommunityForensics), confirming that generator diversity in training data outweighs classifier parameterization.
• Prior-dependent covariance geometry: While homoscedasticity (Mah-NCM) is optimal across most diffusion priors, the reconstruction-based DRCT series exclusively favors heteroscedasticity (QDA with 91.09% vs. 83.25%), reflecting distinct geometric symmetry induced by reconstruction pipelines.
• High sample efficiency on strong encoders: On large foundation representations, Mah-NCM achieves 90.89% CA using just 219 support samples (0.005%). At an extreme 44 samples, diagonal GNB provides the most stable performance (87.14%) due to minimal parameter variance.
• Representation-conditioned Wasserstein-2 metrics: Feature-space distribution shift distance \(\mathcal{W}_2\) displays negligible correlation with actual detector accuracy changes \(\Delta \text{CA}\) across different backbones (Spearman \(\rho = -0.150\) between CF-224 and CF-384; \(\rho = -0.194\) between PE-Core and CF-384). Global distributional divergence often reflects anisotropic embedding scaling rather than deterioration of the decision boundary.

Highlights & Insights

• Demystifying classifier head complexity: The Gaussian ladder provides an analytical baseline that clarifies whether novel detection heads capture genuine non-linear boundaries or merely re-learn second-order geometry with added optimization noise.
• Decoupled system-level diagnostics: Replacing monolithic evaluations with the (prior, encoder, head) triad isolates the exact origins of transfer success, preventing false attribution of representation gains to classifier architecture.
• Zero-shot potential of vision foundation models: Without any anti-forgery fine-tuning, frozen general-purpose vision backbones (PE-Core-bigG) paired with closed-form Mahalanobis distance reach 94.46% CA across 39 benchmarks, vastly exceeding prior specialized detectors.

Limitations & Future Work

• Breakdown under non-Gaussian distributions: The ladder relies on unimodal Gaussian assumptions. In settings with sharp frequency artifacts or heavy lossy compression, feature distributions become multimodal or heavy-tailed, necessitating deep non-linear heads.
• Scalability to multi-class and incremental scenarios: The formulation targets binary real/fake discrimination over a static support prior. Online recursive covariance updates for streaming generative engines remain an open direction.
• Taxonomy Recommendation: The paper was tentatively assigned to llm_nlp due to broad generative keyword tagging. Because it investigates visual diffusion generators, image forensic transfer, and generative image detection, it is recommended to reclassify this paper under ai_safety or aigc_detection.

Related Work & Insights

• vs. UnivFD (Ojha et al., CVPR 2023): UnivFD trains a linear probe on frozen CLIP ViT-L features. The matched audit reveals that closed-form Mah-NCM achieves 57.62%, outperforming UnivFD's 54.63% trained head, proving the limitation lies in empirical risk minimization rather than feature separability.
• vs. AIDE (Yan et al., 2024): AIDE integrates multi-expert frequency and spatial representations. Evaluated on CNNSpot and GenImage, AIDE lags behind the Gaussian ladder by 6–9%, indicating that handcrafted multi-expert fusion can degrade geometric alignment.
• vs. CommunityForensics (Park et al., CVPR 2025): CF demonstrates that generator scaling improves generalization, with CF-384 outperforming Mah-NCM (87.54% vs 84.45%). However, pairing the untuned PE-Core foundation encoder with Mah-NCM reaches 94.46%, showing that foundation model scale surpasses dedicated forensic fine-tuning.

Rating

• Novelty: ⭐⭐⭐⭐ [Repurposes classical Gaussian discriminant analysis into a rigorous diagnostic audit for AIGI generalization]
• Experimental Thoroughness: ⭐⭐⭐⭐⭐ [Evaluated over 39 public benchmarks and 7.1M images, sweeping priors, encoders, sample efficiency, and shift metrics]
• Writing Quality: ⭐⭐⭐⭐⭐ [Clear mathematical grounding, structured argumentation, and informative visualizations]
• Value: ⭐⭐⭐⭐⭐ [Provides an essential, hard-to-beat diagnostic baseline that challenges the necessity of complex classifier head tuning]

Related Papers

• [ECCV 2026] GRE-Diff: Gaussian Room Embeddings for Structured Layout Diffusion
• [ECCV 2026] SAM2Matting: Generalized Image and Video Matting
• [ACL 2025] Stress-testing Machine Generated Text Detection: Shifting Language Models Writing Style to Fool Detectors
• [ICML 2026] Preregistration for Experiments with AI Agents
• [ICML 2026] Token-Efficient Change Detection in LLM APIs