PA-VAD: Purely Pseudo-Anomaly Video Anomaly Detection Based on Diffusion Models¶

Conference: ECCV2026
arXiv: 2512.06845
Code: None
Area: Video Understanding
Keywords: Video Anomaly Detection, Pseudo-Anomaly Generation, Diffusion Models, Domain Alignment, Memory Module

TL;DR¶

PA-VAD proposes a framework completely free of real anomaly videos. It leverages CLIP-guided initial frame selection and VLM prompt refinement to drive video diffusion models to synthesize class-aware pseudo-anomaly segments. Furthermore, a Domain Alignment Regularization Module (DARM) is designed to suppress the magnitude bias of pseudo-anomalies in the feature space. PA-VAD outperforms UVAD SOTA on ShanghaiTech (98.2% AUC), UCF-Crime (82.5%), and XD-Violence (95.1%), even surpassing some WVAD methods that utilize real anomaly videos.

Background & Motivation¶

The goal of video anomaly detection (VAD) is to automatically identify abnormal events, such as assaults, road accidents, and thefts, from surveillance videos. Existing methods are divided into two main paradigms: unsupervised methods (UVAD) and weakly supervised methods (WVAD). UVAD methods are trained solely on normal data and identify anomalies through reconstruction errors or prediction discrepancies; however, they suffer from high false-positive rates under distribution shift and exhibit weak scene generalization. WVAD methods rely on video-level labeled anomaly clips to train MIL detectors and achieve better performance. Nonetheless, collecting real anomaly videos at scale is extremely costly and constrained by security and privacy concerns—many anomalous events are inherently rare and difficult to gather systematically. Both paradigms have their limitations, sharing a key challenge: either they can only detect known normal patterns (UVAD), or they require expensive anomalous data (WVAD).

To alleviate the scarcity of anomaly data, an intuitive direction is to synthesize pseudo-anomalies. Existing works either mix synthesized clips with real anomalies for training (e.g., GVVAD), thereby retaining a dependence on real anomaly videos, or rely on heuristic editing or narrow generative priors, yielding limited diversity and realism. The core challenge lies in whether it is possible to completely eliminate the reliance on real anomaly videos and train a detector solely on synthesized data that matches or even outperforms real-data-driven solutions.

The key finding of this paper is that although pseudo-anomalies synthesized by diffusion models are realistic at the pixel level, they exhibit a significant spatiotemporal magnitude bias in the feature space—the feature norm of pseudo-anomaly segments can be over 20 times larger than that of real anomalies. If these features are directly fed into standard MIL pipelines, the Top-k selection will bias towards a few high-norm instances, causing the detector to learn shortcuts rather than true anomaly semantics. Based on this insight, the Core Idea: PA-VAD is proposed. It drives an I2V diffusion model to synthesize high-quality pseudo-anomaly videos via a Class-Aware Pseudo-Anomaly Generator (CA-PAG) using CLIP frame selection and VLM prompt refinement. Additionally, it designs a Domain Alignment Regularization Module (DARM) that combines domain adversarial alignment and demand-rate memory updates to eliminate the magnitude bias of pseudo-anomalies from both global statistics and local prototype perspectives. This achieves the first weakly supervised VAD completely independent of real anomaly videos, matching or exceeding the performance of real-data WVAD solutions.

Method¶

Overall Architecture¶

The training pipeline of PA-VAD consists of two stages. The first stage is offline pseudo-anomaly generation: initial frames are selected from a database of real normal videos and filtered based on CLIP visual-textual similarity to obtain seed frames that semantically match the target anomaly categories. Then, a VLM generates refined anomaly prompts based on the scenes in the seed frames, driving the Wan2.2 image-to-video (I2V) diffusion model to synthesize pseudo-anomaly clips. The second stage is detector training: real normal videos and synthesized pseudo-anomaly videos are mapped through a feature extractor and fed into the UR-DMU backbone. There, two regularizations of DARM are applied: domain alignment to reduce statistical discrepancies between real and pseudo distributions, and demand-rate memory updates to prevent prototype collapse onto a few high-norm slots. Finally, frame-level anomaly scores are output via MIL ranking loss.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Normal Image Pool"] --> B["CLIP Seed Selection<br/>Visual-Textual Similarity<br/>+ Scene Balancing"]
    B --> C["VLM Prompt Refinement<br/>Scene -> Anomaly Description"]
    C --> D["Wan2.2 I2V<br/>Pseudo-Anomaly Video"]
    D --> E["Detector Features"]
    F["Real Normal Video"] --> G["Detector Features"]
    G --> H["DARM Domain Alignment<br/>GradReverse Adversarial<br/>+ Feature Distance"]
    E --> H
    E --> I["DARM On-Demand Update<br/>Utility-Weighted Prototypes"]
    H --> J["UR-DMU<br/>MIL Detection Head"]
    I --> J
    J --> K["Frame-level Anomaly Scores"]

Key Designs¶

1. CLIP-Guided Initial Frame Selection: Anchoring Generation to Class-Relevant Scenes

Randomly sampling initial frames from the normal video pool easily leads to semantic misalignment—for instance, selecting an indoor home scene for the "Road Accident" class. CA-PAG uses CLIP to score each frame in the joint visual-textual space: a positive query is constructed by concatenating the class name with positive phrases ("surveillance style", "CCTV"), and negative phrases ("black screen", "logo", "dashboard UI") are used to suppress non-surveillance distractions. The final score is computed as \(s(I, c) = \langle\hat{v}(I), \hat{t}(\text{pos})\rangle - \lambda \langle\hat{v}(I), \hat{t}(\text{neg})\rangle\). To prevent high-density camera views (e.g., a massive number of frames captured by the same camera) from dominating the seed selection, a scene-balancing strategy is introduced: the budget is distributed proportionally to the count of each camera scene \(\text{count}(s)^\alpha\), ensuring frames from different view angles are balanced in the Top-K selection. This step is crucial—ablation results show that switching from random selection (86.7% AUC) to CLIP seed selection directly boosts performance by 8.2 percentage points (94.9%).

2. VLM-Driven Prompt Refinement: Translating Coarse Class Names into Scene-Aware Anomaly Prompts

Using only original class names as prompts for the diffusion model (e.g., "Generate fighting") is semantically ambiguous, frequently leading to artifacts like objects disappearing/merging or unnatural actions. CA-PAG uses the Qwen3 30B-A3B VLM to analyze the visual content (objects, layout, illumination) of each seed frame and outputs a concise, scene-consistent description of the anomaly. For example, it refines "Generate burglary" to "a person reaches behind a store counter to rummage through items while another stands watching from the front". The refined prompts are then concatenated with templates such as static camera and natural motion, and fed into the Wan2.2 I2V model (832×480, 81 frames, 25 sampling steps) to synthesize pseudo-anomaly clips. This refinement significantly improves generation quality: FVD drops from 701 to 604, and KVD drops from 57.4 to 34.7, with temporal continuity and physical plausibility markedly enhanced.

3. Domain Alignment Regularization: Eliminating Magnitude Bias from the Global Distribution

Synthesized pseudo-anomalies tend to contain excessive motion, physically implausible frame transitions, and temporal inconsistencies, causing their \(L_2\) norm in the detector feature space to inflate dramatically—for instance, on ShanghaiTech, the average norm of pseudo-anomaly features is 199.48, compared to only 9.13 for real anomalies (over a 20-fold difference). This directly interferes with the Top-k sorting in MIL, letting high-norm pseudo-instances dominate the loss. DARM introduces a DANN-style domain alignment: a gradient reversal layer and a domain discriminator \(D\) are employed to force the feature encoder to be incapable of distinguishing mean features between real normal and pseudo normal streams, while exerting an explicit \(L_2\) feature distance constraint. The effect of domain alignment is global—it reduces the pseudo-anomaly norm from 199.48 to 183.50 (partial contraction), but relying solely on this is insufficient.

4. Demand-Rate Memory Update: Balancing Anomaly Prototype Coverage to Drive Open-Set Generalization

Domain alignment adjusts the global statistics at the normal frame level, but the activation patterns of anomaly memory slots remain uncorrected: a small portion of high-norm slots is repeatedly activated, while low-utility slots are rarely updated. DARM maintains a learnable anomaly memory bank \(M_A\) (\(K\) \(d\)-dimensional prototypes) and calculates the average utility rate \(u_k\) of each slot being activated via soft assignment in the current batch. It then exerts greater attraction on slots with low utility, pulling them toward their responsibility-weighted center \(\mu_k\):

\[\mathcal{L}_{\text{upd}} = \frac{1}{K} \sum_k \left( \frac{\bar{u}}{u_k + \epsilon} \right)^\beta \cdot \|m_k - \mu_k\|_2^2\]

The lower the utility rate, the smaller the denominator \(u_k\), leading to a larger weight and a faster update step. This mechanism has two effects: first, it fundamentally restores the norm of pseudo-anomalies to the level of real anomalies (SHT: 8.39 with full DARM vs. 9.13 for real anomalies); second, it prevents the memory from being monopolized by a few seen-class-specific prototypes, which explains why DARM excels in open-set generalization (improving AUC by 16.6 percentage points compared to OpenVAD on XD with only 1 seen class).

Loss & Training¶

The overall loss is formulated as \(\mathcal{L} = \mathcal{L}_{\text{UR-DMU}} + \lambda_1 \mathcal{L}_{\text{DA}} + \lambda_2 \mathcal{L}_{\text{upd}}\). Where \(\mathcal{L}_{\text{UR-DMU}}\) denotes the MIL ranking loss and uncertainty control loss from the UR-DMU baseline; \(\mathcal{L}_{\text{DA}}\) includes the domain adversarial BCE loss and the \(L_2\) distance of real/pseudo normal features; and \(\mathcal{L}_{\text{upd}}\) represents the utility-weighted memory update loss. Hyperparameters are set as: \(\lambda_1 = 1.0\), \(\lambda_2 = 0.1\), \(\beta = 1.0\), GRL strength \(\lambda_{\text{da}} = 0.1 \sim 0.2\), using the Adam optimizer with a learning rate of \(1e-4 \sim 1e-5\).

Key Experimental Results¶

Main Results¶

Dataset	Metric	PA-VAD (Real/Pseudo)	Prev. UVAD SOTA	Prev. WVAD Real/Real	Comparison
ShanghaiTech	AUC(%)	98.2	MGSTRL 87.5	CMRL 97.6	+0.6 vs. Real Anomaly WVAD
UCF-Crime	AUC(%)	82.5	MGSTRL 80.6	UR-DMU 87.0	+1.9 vs. UVAD
XD-Violence	AUC(%)	95.1	—	UR-DMU 94.2	+0.9 vs. Real Anomaly WVAD

PA-VAD significantly outperforms prior UVAD SOTA across all three datasets and even exceeds WVAD methods trained on real anomaly videos on SHT and XD. There remains a gap on UCF-Crime compared to the Real/Real WVAD configuration, which the authors attribute to the difficulty of current I2V models in high-fidelity synthesis of complex anomalies with long temporal dependencies.

Ablation Study¶

Configuration	SHT AUC	Explanation
Random initial frames	86.7	Baseline, without CLIP selection
+ CLIP seed selection	94.9	+8.2, anchored to class-relevant scenes
+ VLM prompt refinement	96.0	+1.1, full CA-PAG pipeline
+ Domain alignment (DA)	96.8	Part of DARM (global statistical alignment)
+ Utility update (Update)	97.6~97.7	Main driver of DARM
Full DARM (DA+Update)	98.2	Complementary regularizations

Key Findings¶

Utility-aware memory update is the core performance driver of DARM: simply adding "Update" brings an improvement of 1.6 to 1.7 percentage points (96.0 -> 97.6), whereas DA contributes only an additional 0.5 to 0.6 points on top of it. The two regularizations are complementary—DA aligns global statistics, while "Update" corrects local prototype monopolization.
Magnitude bias is a real issue rather than an illusion: the feature norm of pseudo-anomalies in the detector space inflates to 199.48 (SHT), which is 21.8 times that of real anomalies (9.13). The full DARM restores it to 8.39 (\(\approx\) real level), whereas domain alignment alone only reduces it to 183.50—indicating that the utility update is the primary driver of correction on the anomaly side.
Synthesis quantity effect: as the number of clips scale from 14 to 140, the AUC monotonically improves from 85.4% to 98.2%. It already reaches 96.9% with 70 clips (\(\approx\) the 63 real anomaly clips in SHT), and slightly saturates beyond 140 (97.7%).
Significant lead in open-set generalization: when trained with only 1 seen class, the AUC on XD reaches 89.08% (compared to 72.50% for OpenVAD), representing a gain of 16.6 percentage points. This is attributed to the utility update preventing the memory from over-specializing on seen-class prototypes.

Highlights & Insights¶

The goal of "no real anomalies" is strictly achieved for the first time: not as a supplement or a blend, but as a complete replacement. This goes further than works like GVVAD, which only use synthesized data to supplement real data, paving the way for zero-collection-cost VAD.
The discovery and quantification of magnitude bias is a significant contribution: this bias is precisely quantified—a 20-fold difference in feature norm occurs consistently across multiple backbones (I3D/Qwen/C3D), indicating that it is an inherent property of synthesized data rather than a model-specific artifact.
The design of the utility-weighted update is elegant and transferable: the weight \((\bar{u} / (u_k + \epsilon))^\beta\) provides a clear intuition—slots with higher usage undergo smaller updates, while those with lower usage are updated more aggressively. This idea can be generalized to any MIL or metric learning scenario requiring balanced prototype coverage.
DARM simultaneously addresses closed-set and open-set scenarios: domain alignment improves closed-set performance, while the utility update drives open-set generalization. A unified mechanism covers both dimensions.

Limitations & Future Work¶

A performance gap still exists on UCF-Crime compared to the Real/Real WVAD SOTA (82.5% vs. UR-DMU 87.0%). This is primarily because complex anomalies with long-term dependencies (e.g., the full process of theft: "approach \(\rightarrow\) observe \(\rightarrow\) take \(\rightarrow\) leave") are difficult to synthesize with high fidelity using current I2V models—rendering generation quality a bottleneck.
High generation cost: generating a single clip with Wan2.2 takes about 9.5 minutes (on dual RTX 6000 Ada GPUs), and the entire SHT generation takes around 30 hours. Actual deployment requires trade-offs between time and computing power.
Reliance on large-model priors: CA-PAG invokes three pre-trained models—CLIP, VLM, and Video Diffusion. Although this bypasses real anomaly collection, the generation quality may degrade if the target scenario deviates significantly from the pre-training distributions.
Prior category knowledge is a prerequisite: the names of anomalous classes must be known in advance to generate the corresponding pseudo-anomalies. True zero-shot VAD without any prior knowledge remains unresolved.

vs. GVVAD (Cai et al.): Both utilize diffusion models to synthesize pseudo-anomalies. However, GVVAD mixes synthesized clips with real anomalies for training, whereas PA-VAD completely replaces real anomalies and systematically resolves the magnitude bias.
vs. OpenVAD (Zhu et al.): OpenVAD also targets open-set VAD but requires real anomalies during training. PA-VAD achieves stronger open-set generalization using only synthesized pseudo-anomalies, demonstrating that high-quality pseudo-anomalies combined with regularized training can indeed replace real data.
vs. UR-DMU: UR-DMU serves as the detector backbone for PA-VAD (already featuring dual memory banks and an MIL head), but direct training on pseudo-anomalies leads to severe degradation due to magnitude bias. DARM introduces domain alignment and utility updates on top of it, boosting the SHT baseline from 96.0% to 98.2%.
vs. Wan2.2 / Stable Video Diffusion: These I2V diffusion models are not natively designed for anomaly synthesis. The contribution of CA-PAG lies in combining CLIP-based frame selection and VLM-based prompt refinement to adapt general I2V models to the anomaly synthesis task, thereby systematizing and automating prompt engineering.

Rating¶

Novelty: ⭐⭐⭐⭐☆ Promoted pseudo-anomalies from "supplementary" to "replacement", systematically discovery and settlement of magnitude bias, overall novel philosophy.
Experimental Thoroughness: ⭐⭐⭐⭐⭐ Comprehensive coverage across three benchmarks, open-set protocols, detailed ablations, magnitude bias quantification, and synthesis quantity analysis.
Writing Quality: ⭐⭐⭐⭐☆ Clear motivation and problem definition, structured method presentation, though detailed equations and training specifics are mostly relegated to supplementary materials.
Value: ⭐⭐⭐⭐⭐ Drastically lowers the data barrier for VAD deployment with outstanding open-set generalization, directly driving practical security applications.