PA-VAD: Pure Pseudo-Anomalous Video Anomaly Detection Based on Diffusion Models¶

Conference: ECCV2026
arXiv: 2512.06845
Code: None
Area: Video Understanding
Keywords: Video Anomaly Detection, Pseudo-Anomaly Generation, Diffusion Models, Domain Alignment, Memory Module

TL;DR¶

PA-VAD proposes a framework completely free of any real anomalous videos. By leveraging CLIP-guided initial frame selection and VLM prompt refinement, it drives a video diffusion model to synthesize class-aware pseudo-anomalous segments. It designs a Domain Alignment Regularization Module (DARM) to suppress the magnitude bias of pseudo-anomalies in the feature space. PA-VAD outperforms UVAD SOTA on ShanghaiTech (98.2% AUC), UCF-Crime (82.5%), and XD-Violence (95.1%), even surpassing some WVAD methods that utilize real anomalous videos.

Background & Motivation¶

The objective of Video Anomaly Detection (VAD) is to automatically identify anomalous events, such as assaults, traffic accidents, and thefts, from surveillance videos. Existing methods fall into two paradigms: Unsupervised VAD (UVAD) trains only on normal data and identifies anomalies based on reconstruction errors or prediction discrepancies, yet suffers from high false-alarm rates under distribution shifts and poor scene generalization; Weakly Supervised VAD (WVAD) utilizes video-level labels of anomalous segments to train MIL detectors and yields better performance, but acquiring large-scale real anomalous videos is highly expensive and restricted by safety and privacy policies—many anomaly categories are rare and hard to collect systematically. While both paradigms face unique limitations, they share a key challenge: either being constrained to detecting only known normal patterns (UVAD) or requiring expensive anomalous data (WVAD).

To alleviate the scarcity of anomalous data, an intuitive approach is to synthesize pseudo-anomalies. Existing studies either mix synthetic segments with real anomalies for training (e.g., GVVAD), which still retains reliance on real anomalous videos, or rely on heuristic editing/narrow generative priors, limiting synthesis diversity and realism. The core challenge becomes: can we completely eliminate the reliance on real anomalous videos, and train a detector solely on synthetic data that matches or even outperforms real-data-based paradigms?

The key finding of this paper is that although pseudo-anomalies synthesized by diffusion models are realistic at the pixel level, they exhibit a significant spatiotemporal magnitude bias in the feature space—the feature norm of pseudo-anomalous segments can be more than 20 times that of real anomalies. If directly fed into a standard MIL pipeline, the Top-k selection will be biased towards a few high-norm instances, causing the detector to learn shortcuts rather than genuine anomaly semantics. Based on this insight, the Core Idea is: propose PA-VAD, which drives an I2V diffusion model to synthesize high-quality pseudo-anomalous videos through a Class-Aware Pseudo-Anomaly Generator (CA-PAG) using CLIP frame selection and VLM prompt writing. It also designs a Domain Alignment Regularization Module (DARM) that combines domain adversarial alignment and demand-aware usage-rate memory updates to eliminate pseudo-anomaly magnitude bias from both global statistics and local prototypes. This achieves weakly supervised VAD completely without real anomalous videos for the first time, reaching performance that surpasses real-data-based WVAD schemes.

Method¶

Overall Architecture¶

The training pipeline of PA-VAD consists of two stages. The first stage is offline pseudo-anomaly generation: initial frames are selected from a pool of real normal videos, and those with semantic matches to target anomaly classes are filtered as seed frames via CLIP visual-textual similarity. A VLM then generates refined anomaly prompts based on the scenes of the seed frames to drive the Wan2.2 image-to-video diffusion model to synthesize pseudo-anomalous segments. The second stage is detector training: real normal videos and synthesized pseudo-anomalous videos are passed through a feature extractor to the UR-DMU backbone. Two regularizations from DARM are applied—domain alignment reduces the statistical discrepancy between real/pseudo distributions, and demand-aware usage-rate memory updates prevent prototype concentration on a few high-norm slots. Finally, frame-level anomaly scores are outputted via the MIL ranking loss.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Normal Image Pool"] --> B["CLIP Seed Selection<br/>Visual-Textual Similarity<br/>+ Scene Balancing"]
    B --> C["VLM Prompt Refinement<br/>Scene -> Anomaly Description"]
    C --> D["Wan2.2 I2V<br/>Pseudo-Anomalous Video"]
    D --> E["Detector Features"]
    F["Real Normal Video"] --> G["Detector Features"]
    G --> H["DARM Domain Alignment<br/>GradReverse Adversarial<br/>+ Feature Distance"]
    E --> H
    E --> I["DARM Demand-based Update<br/>Usage-weighted Prototype"]
    H --> J["UR-DMU<br/>MIL Detection Head"]
    I --> J
    J --> K["Frame-level Anomaly Score"]

Key Designs¶

1. CLIP-Guided Initial Frame Selection: Anchoring Generation to Class-Relevant Scenes

Randomly sampling initial frames from a normal video pool easily leads to semantic mismatch—for instance, selecting an indoor home scene for the "Road Accident" class. CA-PAG scores each frame in a joint visual-textual space using CLIP: positive queries are constructed by appending positive phrases (surveillance style, CCTV) to class names, while negative phrases (black screen, logo, dashboard UI) are used to suppress non-surveillance interference, yielding the final score as s(I, c) = ⟨v̂(I), t̂(pos)⟩ − λ ⟨v̂(I), t̂(neg)⟩. To prevent high-density camera views (e.g., a massive number of frames captured by the same camera) from dominating seed selection, a scene-balancing strategy is introduced: the budget is distributed proportionally to the count of each camera scene count(s)^α, ensuring frames from different views are balanced to enter the Top-K. This step is critical—ablation shows that switching from random selection (86.7% AUC) to CLIP seed selection directly boosts performance by 8.2 percentage points (94.9%).

2. VLM-driven Prompt Refinement: Translating Coarse Class Names to Scene-Aware Anomaly Texts

Relying solely on dry class names as diffusion model prompts (e.g., "Generate fighting") leads to vague semantics; generated outcomes often suffer from artifacts like disappearing/merging objects and unnatural motion. CA-PAG uses Qwen3 30B-A3B VLM to analyze the visual content (objects, layout, illumination) of each seed frame to output a short, scene-consistent anomaly description. For example, it refines "Generate burglary" to "a person reaches behind a store counter to rummage through items while another stands watching from the front." The refined prompts are then concatenated with templates like static cameras and natural motions, and fed into the Wan2.2 I2V model (832x480, 81 frames, 25 sampling steps) to synthesize pseudo-anomalous segments. This refinement significantly improves generation quality: FVD drops from 701 to 604, KVD from 57.4 to 34.7, and temporal continuity and physical plausibility are noticeably improved.

3. Domain Alignment Regularization: Eliminating Magnitude Bias from the Global Distribution

Synthesized pseudo-anomalies tend to contain excessive motion, physically implausible frame transitions, and temporal inconsistencies, causing their features in the detector's space to balloon drastically in terms of \(\ell_2\) norm—for instance, on ShanghaiTech, the average norm of pseudo-anomaly features is 199.48, whereas real anomalies are only 9.13 (a difference of over 20 times). This directly interferes with the Top-k ordering of MIL, allowing high-norm pseudo-instances to dominate the loss. DARM introduces DANN-style domain alignment: a Gradient Reversal Layer combined with a domain discriminator D forces the feature encoder to be incapable of distinguishing mean features between the real Normal and pseudo-Normal streams, while applying an explicit feature distance L2 constraint. The effect of domain alignment is global—it reduces the pseudo-anomaly norm from 199.48 to 183.50 (a partial reduction), but relying on it alone is insufficient.

4. Demand-Based Memory Update: Balancing Anomaly Prototype Coverage to Drive Open-Set Generalization

Domain alignment adjusts global statistics at the normal frame level, but the activation patterns of anomaly memory slots still remain uncorrected: a few high-norm slots are repeatedly activated, while low-usage slots are rarely updated. DARM maintains a learnable anomaly memory bank M_A (K d-dimensional prototypes) and calculates the average usage rate u_k of each slot being activated via soft assignment in the current batch. Then, it applies a larger pulling force to low-usage slots to pull them toward the responsibility-weighted center µ_k:

ℒ_upd = (1/K) Σ_k (ū / (u_k+ε))^β · ||m_k − µ_k||²₂

The lower the usage rate, the smaller the denominator u_k, the larger the weight, and the faster the update step. This mechanism has two effects: first, it fundamentally restores the norm of pseudo-anomalies to the level of real anomalies (SHT: 8.39 after complete DARM vs 9.13 for real); second, it prevents the memory from being monopolized by a few seen-class-specific prototypes—which is why DARM performs exceptionally well in open-set generalization (with only one seen class, the AUC on XD is 16.6 percentage points higher than OpenVAD).

Loss & Training¶

The total loss is ℒ = ℒ_UR-DMU + λ₁ℒ_DA + λ₂ℒ_upd. ℒ_UR-DMU is the MIL ranking loss and uncertainty control loss of the UR-DMU baseline; ℒ_DA includes the domain adversarial BCE loss and Normal-pseudo Normal feature distance L2; ℒ_upd is the usage-weighted memory update loss. Hyperparameters: λ₁=1.0, λ₂=0.1, β=1.0, GRL intensity λ_da=0.1~0.2, Adam optimizer, learning rate 1e-4~1e-5.

Key Experimental Results¶

Main Results¶

Dataset	Metric	PA-VAD (Real/Pseudo)	Prev. UVAD SOTA	Prev. WVAD Real/Real	Comparison
ShanghaiTech	AUC(%)	98.2	MGSTRL 87.5	CMRL 97.6	+0.6 vs Real-anomaly WVAD
UCF-Crime	AUC(%)	82.5	MGSTRL 80.6	UR-DMU 87.0	+1.9 vs UVAD
XD-Violence	AUC(%)	95.1	—	UR-DMU 94.2	+0.9 vs Real-anomaly WVAD

PA-VAD significantly outperforms UVAD SOTA across all three datasets and even excels over WVAD methods using real anomaly videos on SHT and XD. There is still a gap compared to Real/Real WVAD on UCF-Crime (which the paper attributes to the presence of more complex anomalies with long-term dependencies in this dataset, which are difficult for current I2V models to synthesize with high fidelity).

Ablation Study¶

Configuration	SHT AUC	Description
Random initial frames	86.7	Baseline, without CLIP selection
+ CLIP seed selection	94.9	+8.2, anchored to class-relevant scenes
+ VLM prompt refinement	96.0	+1.1, full CA-PAG
+ Domain Alignment (DA)	96.8	DARM component (global statistical alignment)
+ Usage-rate Update (Update)	97.6~97.7	Primary driver of DARM
Full DARM (DA+Update)	98.2	Two regularizations are complementary

Key Findings¶

Usage-aware memory update is the performance core of DARM: adding Update alone brings a 1.6-1.7 percentage point improvement (96.0 -> 97.6), while DA contributes only an additional 0.5-0.6 points on top of this. The two regularizations are complementary—DA aligns global statistics, and Update corrects local prototype monopoly.
Magnitude bias is a real issue rather than an illusion: the feature norm of pseudo-anomalies in the detector space inflates to 199.48 (SHT), which is 21.8 times that of real anomalies (9.13). The full DARM restores it to 8.39 (approx. real level), while domain alignment alone only reduces it to 183.50—indicating that the usage-rate update is the primary force for correction on the anomaly side.
Synthesis quantity effect: from 14 segments to 140 segments, the AUC monotonically increases from 85.4% to 98.2%. It already reaches 96.9% at 70 segments (nearly equivalent to SHT's 63 real anomalies) and slightly saturates after exceeding 140 (97.7%).
Significant lead in open-set generalization: when trained with only 1 seen class, the AUC on XD reaches 89.08% (compared to 72.50% of OpenVAD), representing a 16.6 percentage point gain. This is due to the usage-rate update preventing the memory from over-specializing on seen-class prototypes.

Highlights & Insights¶

The goal of "no real anomalies" is strictly achieved for the first time: not as a supplement or a blend, but as a complete replacement. This goes a step further than works like GVVAD that only treat synthesis as a supplement to real anomalies, opening the door to zero-collection-cost VAD.
The discovery and quantification of magnitude bias is a significant contribution: this bias is precisely quantified—a 20-fold feature norm gap that consistently appears across multiple backbones (I3D/Qwen/C3D), indicating it is an intrinsic property of synthesized data rather than a model-specific artifact.
The design of usage-weighted updates is clever and transferable: the intuition behind the weight (ū / (u_k+ε))^β is very clear—the more a slot is used, the less it is updated; the less it is used, the tighter it is pulled. This idea can be generalized to any MIL or metric learning scenario requiring balanced prototype coverage.
DARM handles both closed-set and open-set scenarios: domain alignment improves closed-set performance, while the usage-rate update drives open-set generalization, covering both dimensions with a single joint mechanism.

Limitations & Future Work¶

Still lagging behind Real/Real WVAD SOTA on UCF-Crime (82.5% vs UR-DMU 87.0%), primarily because complex anomalies with long-term dependencies (such as the entire process of burglary: "approach -> observe -> take -> leave") are difficult to synthesize with high fidelity by current I2V models, making generation quality a bottleneck.
High generation cost: generating one segment using Wan2.2 takes about 9.5 minutes (on dual RTX 6000 Ada GPUs). Synthesizing the whole set for SHT takes roughly 30 hours, requiring a trade-off between time and computing power in practice.
Reliance on large model priors: CA-PAG relies on three pre-trained models—CLIP, VLM, and Video Diffusion. Although this replaces real anomaly collection, the generation quality may degrade if the target scenario deviates significantly from the pre-training distribution.
Class knowledge is a prerequisite: class names of anomalies must be known in advance to generate corresponding pseudo-anomalies. True zero-shot prior-free detection remains unsolved.

vs GVVAD (Cai et al.): Both use diffusion to synthesize pseudo-anomalies, but GVVAD trains with a mix of synthetic segments and real anomalies, whereas PA-VAD completely replaces real anomalies and additionally addresses the magnitude bias issue systematically.
vs OpenVAD (Zhu et al.): OpenVAD also conducts open-set VAD but relies on real anomalies in training. PA-VAD achieves stronger open-set generalization using only synthetic pseudo-anomalies, demonstrating that high-quality pseudo-anomalies combined with regularized training can indeed replace real data.
vs UR-DMU: UR-DMU is the detector backbone of PA-VAD (which already includes dual memory banks and an MIL head), but it suffers from severe degradation when trained directly on pseudo-anomalies due to magnitude bias. DARM adds domain alignment and usage-rate updates on top of it, lifting the SHT baseline from 96.0% to 98.2%.
vs Wan2.2 / Stable Video Diffusion: These I2V diffusion models are not natively designed for anomaly synthesis. The contribution of CA-PAG lies in adapting a general I2V model to the anomaly synthesis task through a combination of CLIP frame selection and VLM prompt writing, thereby systematizing and automating prompt engineering.

Rating¶

Novelty: ⭐⭐⭐⭐☆ The elevation of pseudo-anomalies from "supplement" to "replacement" and the systematic discovery of and solution to the magnitude bias represent a highly novel concept.
Experimental Thoroughness: ⭐⭐⭐⭐⭐ Comprehensive coverage with evaluations across three benchmarks, open-set protocols, detailed ablations, quantification of magnitude bias, and synthesis quantity analysis.
Writing Quality: ⭐⭐⭐⭐☆ Clear definition of motivation and problem, hierarchical structuring of methods, though detailed formulations and training specifications are heavily relegated to supplementary materials.
Value: ⭐⭐⭐⭐⭐ Significantly lowers the data acquisition barrier for VAD deployment. The outstanding open-set generalization results directly facilitate practical physical safety and monitoring applications.