DeCo: Zero-Shot Anomaly Generation through Decoupling and Recoupling¶

Conference: ECCV 2026
Paper: CVF / ECCV Poster
Code: https://github.com/HUST-SLOW/DeCo
Area: Others
Keywords: industrial anomaly generation, zero-shot generation, diffusion models, decoupling and recoupling, defect inspection

TL;DR¶

Addressing the scarcity of real industrial defects and the issues of inaccurate anomaly acquisition and uncontrolled fusion in prior work, DeCo decouples pure anomaly structure from source products via DR-Flow and PI-Flow, binding it to an abnormal token, and explicitly recouples it with target product textures via hybrid LoRA injection and Product Compatibility Correction (PCC), significantly advancing downstream detection SOTA on MVTec AD and VisA.

Background & Motivation¶

Industrial anomaly inspection is fundamentally bottlenecked by the acute scarcity of real defect samples in cold-start manufacturing environments. Traditional heuristic anomaly synthesis methods (such as CutPaste, DRAEM, and NSA) rely on random cut-and-paste or handcrafted perturbations, frequently generating pseudo-anomalies that lack structural consistency and physical plausibility. On the other hand, few-shot generative approaches still require a handful of real defect images for each specific product, which proves impractical when deploying inspection models for newly introduced product lines. This has sparked intense interest in zero-shot industrial anomaly generation—aiming to synthesize realistic defects on a target product using only its normal images alongside defect information gathered from disjoint reference products.

However, existing zero-shot anomaly generation approaches suffer from two critical limitations. The first is inaccurate anomaly information acquisition. Text-driven methods (e.g., AnomalyAny, AnoStyler) depend on fixed text descriptions to guide diffusion generation; given the vast semantic gap between generic text prompts and complex, non-rigid industrial deformations, they often synthesize unrealistic or repetitive patterns. Conversely, reference-based methods attempt to extract visual features from other defective products, but struggle to disentangle the defect structure from the source product's background, causing unwanted source colors and surface textures to leak into the generated images. The second limitation is uncontrolled anomaly-product fusion. Current methods typically rely on naive composite text prompts (such as "a cut on carpet") to implicitly steer generation, lacking explicit mechanisms to blend foreign defects into the target product's native textures, which frequently results in visual incompatibility or washed-out artifacts.

This paper tackles these challenges with a clear insight: an industrial anomaly's core identity lies in its spatial topological deformation ("structure") rather than its host product's material appearance ("texture"). Because self-attention mechanisms in diffusion transformers inherently separate the roles of Query (spatial layout) and Key/Value (appearance content), one can decouple structure from texture through carefully orchestrated dual flows. Core idea: completely decouple the anomaly structure from its source product context and bind it to an isolated abnormal text token via DR-Flow and PI-Flow, then explicitly recouple it with the target product's normal texture using hybrid LoRA injection and Product Compatibility Correction (PCC) for high-fidelity zero-shot anomaly generation.

Method¶

Overall Architecture¶

DeCo is constructed upon the pretrained Stable Diffusion 3 (SD3) Multimodal Diffusion Transformer (MM-DiT) framework driven by Rectified Flow. The overall pipeline consists of two distinct stages: anomaly structure decoupling during training and target product recoupling during inference. In the decoupling stage, the model first pretrains and freezes a product adapter (P-LoRA) on normal images of the source product to anchor normal textures. It then trains an anomaly adapter (A-LoRA) using Dual-Routing Flow (DR-Flow)—where AP-Routing isolates the abnormal text token and QK-Routing binds the image structure query to this token—paired with Product-Invariant Flow (PI-Flow) to subtract the product-specific velocity field. In the recoupling stage, the source A-LoRA and target P-LoRA are fused via hybrid injection, modulated by Product Compatibility Correction (PCC) to push the trajectory away from pure-product velocity, and blended locally with the target normal image.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}, 'subGraphTitleMargin': {'top': 8, 'bottom': 16}}}%%
flowchart TD
    subgraph S1["Stage 1: Anomaly Structure Decoupling"]
        direction TB
        A["Reference Anomaly & Normal Images"] --> B["Dual-Routing Flow DR-Flow<br/>AP token split + QK structure routing"]
        A --> C["Product-Invariant Flow PI-Flow<br/>Residual velocity constraint against leakage"]
        B & C --> D["Disentangled A-LoRA<br/>Pure structure without source texture"]
    end
    subgraph S2["Stage 2: Target Product Recoupling & Generation"]
        direction TB
        E["Target Normal Image & Target P-LoRA"] & D --> F["Hybrid LoRA Injection<br/>W_inf = W0 + ΔW_P + ΔW_A"]
        F --> G["Product Compatibility Correction PCC<br/>Extrapolate away from pure-product flow"]
        G --> H["Blended Diffusion Inpainting<br/>High-fidelity synthetic anomaly on target"]
    end

Key Designs¶

1. Dual-Routing Flow (DR-Flow): explicitly binding anomaly structure via AP and QK routing Because industrial anomalies lack fine-grained concept priors in pre-trained language models, standard joint fine-tuning easily absorbs the entire reference image (both background and defect) into the token embedding. DR-Flow establishes two orthogonal routing paths. On the text side, AP-Routing decomposes the composite prompt into the abnormal token $T_A$, product token $T_P$, and auxiliary tokens $T_O$. $T_P$ is exclusively projected through the frozen P-LoRA while $T_A$ is passed solely through the trainable A-LoRA, preventing any parameter interference between anomaly and product descriptions. On the image side, QK-Routing leverages the structural role of Query and the textural role of Key/Value: it routes the image token $T_I$ through frozen P-LoRA weights to form Keys and Values ($K_I, V_I$), restricting the available visual appearance strictly to normal textures, while routing $T_I$ through trainable A-LoRA to form Queries ($Q_I$). Through cross-modal self-attention, A-LoRA is forced to bind solely the spatial topological structure to the isolated abnormal token under the flow-matching objective within the masked anomaly region $m$: $$ \mathcal{L}{\text{DR}} = \mathbb{E}) - v_t) |_2^2 \right] $$ where } \left[ | m \odot (\hat{v}_{\text{DR}}(z_t, t, \mathcal{P$v_t = z_1 - z_0$ represents the constant ground-truth velocity field, and $\hat{v}_{\text{DR}}$ is the predicted velocity.

2. Product-Invariant Flow (PI-Flow): residual velocity constraint eliminating background leakage Although QK-Routing strips explicit Key/Value texture updates, A-LoRA can still exploit implicit shortcut features during gradient back-propagation to memorize source product colors and reduce loss. PI-Flow prevents this leakage by imposing a parallel dynamical constraint. When fed with the source product prompt and noise latent $z_t$, the frozen P-LoRA alone predicts a product-only velocity field $\hat{v}_p$ that reconstructs normal backgrounds. Subtracting this product velocity from the target velocity yields a pure residual anomaly flow $v_{\text{PI}} = v_t - \hat{v}_p$. Training A-LoRA alone on the anomaly prompt $P_a$ to predict $\hat{v}_{\text{PI}}$ penalizes any residual correlation with source product textures: $$ \mathcal{L}{\text{PI}} = \mathbb{E}} \left[ | m \odot (\hat{v{\text{PI}}(z_t, t, \mathcal{P}_a) - v) |_2^2 \right] $$ The joint training objective is }$\mathcal{L}_{\text{total}} = \mathcal{L}_{\text{DR}} + \lambda_{\text{PI}} \mathcal{L}_{\text{PI}}$ (with $\lambda_{\text{PI}} = 0.1$). This formulation guarantees that A-LoRA learns a product-invariant anomaly representation.

3. Hybrid LoRA Injection with Product Compatibility Correction (PCC): resolving cross-domain incompatibility At test time, to transfer the acquired anomaly structure onto a completely different target product $T$, DeCo injects both target P-LoRA and source A-LoRA into SD3: $W_{\text{inf}} = W_0 + \Delta W_{\mathcal{T}}^{\text{P}} + \Delta W_{\mathcal{R}}^{\text{A}}$. However, because both LoRAs are trained independently across distinct domains, directly recoupling them often triggers severe incompatibility—where the target product's dominant texture overwhelms the anomaly structure, rendering the synthesized defect faint or invisible. Inspired by Classifier-Free Guidance extrapolation, PCC computes the target product-only velocity $\hat{v}_p$ using target P-LoRA alone, and pushes the final inference velocity away from this normal baseline: $$ \hat{v}{\text{inf}} = \hat{v}_p + \omega \cdot (\hat{v}_p) $$ where }} - \hat{v$\omega$ is the correction factor (set to $\omega=2$). This extrapolation sharply enhances the visual contrast and structural presence of the anomaly on the target surface, followed by blended diffusion to restrict modifications strictly within the anomaly mask.

Loss & Training¶

The framework is optimized end-to-end within the Rectified Flow Matching regime: - P-LoRA Pretraining: Independently trained on normal image collections for each product category; backbone weights $W_0$ remain frozen. - A-LoRA Joint Disentanglement: Freezing $W_0$ and P-LoRA, the training samples timesteps $t \sim \mathcal{U}[0, 1]$ and latent paths $z_t = (1-t)z_0 + tz_1$, concurrently optimizing $\mathcal{L}_{\text{DR}}$ and $\mathcal{L}_{\text{PI}}$. - Hyperparameter Setup: $\lambda_{\text{PI}} = 0.1$, inference correction scale $\omega = 2$.

Key Experimental Results¶

Main Results¶

The authors evaluate both visual generation fidelity and downstream anomaly detection/segmentation performance on MVTec AD and VisA datasets (training a standard U-Net on generated image-mask pairs).

Table 1: Quantitative comparison of anomaly image generation quality on MVTec AD (corresponding to Table 1 in paper)

Category	Method	Authenticity (IS ↑)	Anomaly-Region Authenticity (IS(a) ↑)	Diversity (IC-L ↑)
Few-Shot Gen.	DFMGAN [8]	1.72	2.80	0.20
Few-Shot Gen.	AnoDiff [16]	1.80	2.77	0.27
Few-Shot Gen.	AnoGen [19]	1.77	2.84	0.32
Anomaly Synth.	CutPaste [20]	1.76	–	0.22
Anomaly Synth.	DRAEM [37]	1.76	–	0.25
Anomaly Synth.	NSA [29]	1.44	–	0.26
Anomaly Synth.	RealNet [38]	1.64	–	0.22
Zero-Shot Gen.	AnomalyAny [32]	2.02	2.98	0.33
Zero-Shot Gen.	AnoStyler [31]	2.04	3.12	0.32
Ours	DeCo	2.06	3.98	0.33

Table 2: Downstream anomaly detection performance on MVTec AD (corresponding to Table 2 in paper)

Method Type	Method	Image-Level I-AUC (%)	Image-Level I-AP (%)	Pixel-Level P-AUC (%)	Pixel-Level P-AP (%)	PRO (%)
Anomaly Synth.	CutPaste	89.8	92.1	88.2	51.9	76.4
Anomaly Synth.	DRAEM	94.6	97.0	92.2	54.1	83.1
Anomaly Synth.	NSA	93.0	95.6	92.0	52.6	82.2
Anomaly Synth.	RealNet	95.2	97.0	94.0	57.7	85.2
Zero-Shot Gen.	AnomalyAny	95.2	96.9	89.0	62.7	84.7
Zero-Shot Gen.	AnoStyler	98.0	99.0	94.4	62.9	88.3
LoRA Decoupling	UnzipLoRA [22]	96.6	98.5	94.6	61.9	85.0
LoRA Decoupling	QR-LoRA [35]	95.4	97.2	93.1	60.2	84.2
Ours	DeCo	98.0	99.1	95.3	68.0	89.6

Ablation Study¶

The ablation isolates the contribution of each core component on the MVTec AD benchmark.

Table 3: Ablation study of core components (corresponding to Table 4 in paper)

Row	AP-Routing	QK-Routing	PI-Flow	PCC	I-AUC (%)	P-AUC (%)	P-AP (%)	P-F1 (%)
(a)	–	–	–	–	94.0	94.2	62.7	60.5
(b)	✓	–	–	–	95.7	94.3	63.2	61.1
(c)	✓	✓	–	–	96.9	95.0	66.7	63.0
(d)	✓	✓	✓	–	97.5	95.1	67.6	63.2
(e)	✓	✓	✓	✓	98.0	95.3	68.0	64.5

Key Findings¶

Unprecedented local defect authenticity: On the localized anomaly metric IS(a) in Table 1, DeCo scores 3.98, far exceeding the best prior zero-shot method AnoStyler (3.12) and few-shot methods like AnoGen (2.84), proving that decoupled structures blend with physical realism without blur or artificial borders.
Critical role of structural routing and residual flow: As shown in Table 3, introducing QK-Routing drives pixel AP from 63.2% to 66.7% (+3.5%), while PI-Flow further boosts it to 67.6%, proving that preventing source background leakage is decisive for accurate pixel-level localization.
Superior cross-dataset generalization on VisA: When directly transferring anomaly structures acquired on MVTec AD to the 12 complex objects of VisA (Table 3 in paper), DeCo delivers 41.5% P-AP and 46.4% P-F1, outperforming the strongest baseline (AnoStyler at 31.4% P-AP) by an absolute margin of 8.2% P-AP.

Highlights & Insights¶

Repurposing self-attention semantics for industrial disentanglement: By exploiting the finding that Query dictates spatial layout while Key/Value dictate visual appearance, locking Key/Value to normal background P-LoRA forces A-LoRA to focus exclusively on geometric structure.
Dynamical residual velocity formulation (PI-Flow): Instead of relying on static feature orthogonality constraints, PI-Flow formulates decoupling directly within the continuous velocity field of Rectified Flow, effectively subtracting the normal product trajectory from the total flow.
Classifier-Free-like guidance for cross-concept compatibility (PCC): Identifying that cross-product LoRA coupling suffers from visibility fading, PCC extrapolates away from the pure-product velocity vector, providing an elegant, training-free inference solution for multi-LoRA compositional generation.

Limitations & Future Work¶

Dependency on conditional mask priors: DeCo currently assumes access to reference defect masks or shape-augmented bounding boxes to guide localized blended diffusion; generating end-to-end defective layouts without external masks remains an open problem.
Latent compression artifacts on microscopic defects: Because SD3 operates in an $8\times$ downsampled VAE latent space, ultra-fine sub-pixel anomalies (such as microscopic hairline cracks or pinhole punctures) can suffer from soft boundaries or quantization artifacts.
Future directions: Integrating multi-scale latent architectures or pixel-space diffusion stages to capture ultra-high-resolution industrial defects, and jointly generating anomaly masks with contextual semantic layouts.

vs AnoStyler / AnomalyAny (Text-driven zero-shot generation): These rely on broad descriptive text prompts that fail to capture intricate structural anomalies; DeCo extracts precise geometric topologies from reference images, elevating IS(a) from 3.12 to 3.98.
vs UnzipLoRA / QR-LoRA (General-purpose LoRA decoupling): Standard LoRA composition relies on strong pre-trained semantic priors (e.g., separating "dog" from "style"); industrial anomalies lack such priors. DeCo's structural QK-Routing and velocity-level PI-Flow achieve a +6.1% P-AP margin over UnzipLoRA on downstream inspection.

Rating¶

Novelty: ⭐⭐⭐⭐⭐ Elegant exploitation of attention query/key roles and Rectified Flow residual velocity for structure-texture decoupling.
Experimental Thoroughness: ⭐⭐⭐⭐⭐ Comprehensive evaluation covering generation metrics, downstream detection across two major datasets, and step-by-step ablations.
Writing Quality: ⭐⭐⭐⭐⭐ Lucid problem formulation, self-contained mathematical exposition, and crisp diagrams.
Value: ⭐⭐⭐⭐⭐ Offers a practical, high-impact data generation engine for industrial cold-start inspection; fully open-sourced.