DeltaDeno: Zero-Shot Anomaly Generation via Delta-Denoising Attribution

Conference: ECCV2026
Paper: ECCV Paper
Code: https://github.com/CROVO1026/DeltaDeno
Area: Anomaly Detection / Image Generation
Keywords: zero-shot anomaly generation, delta-denoising attribution, prompt refinement, spatial attention biasing, latent inpainting

TL;DR

DeltaDeno drives two synchronized diffusion trajectories with normal and anomaly prompts, accumulates post-denoising latent differences into a defect localization map, and uses that map to constrain local generation, increasing downstream MVTec AD image-level AUROC from AnomalyAny's 77.6 to 84.7 without generator fine-tuning or real anomalous samples.

Background & Motivation

The main difficulty in industrial inspection is not a shortage of normal images, but defects that are rare and continually changing, with pixel-level annotation requiring expert effort. When production materials, lighting, or processes change, previously collected anomalies may no longer represent faults that could emerge on a new production line. Anomaly generation therefore starts from readily available normal images to provide defect images and masks for training and stress testing. Synthesis strategies such as DRAEM transfer readily, but pasted regions can disagree with the original material or lighting; few-shot methods such as AnomalyDiffusion learn from real defects but still require anomalous images and annotations. Collecting anomalies before addressing anomaly scarcity remains a deployment barrier, and category priors learned through fine-tuning may not transfer to unfamiliar objects.

Zero-shot anomaly generation imposes a stricter setting: only normal references, textual defect descriptions, and a pretrained generator are available, without task-specific model adaptation. The objective is not merely to introduce a visual change, but also to identify its location so that the synthetic image provides reliable segmentation supervision. AnomalyAny extracts masks from cross-attention maps, but these maps are coarse, and where a word receives attention need not precisely identify pixels actually changed into defects. A defect prompt can also alter the background or overall object shape, causing unrelated changes to enter the mask. Localization, generation, and background preservation are therefore interdependent: an overly broad region permits irrelevant edits, while an inaccurate mask contaminates downstream training.

This paper instead derives localization from changes within diffusion itself: keep the random seed and schedule fixed, alter only the anomalous attribute in the prompt, and observe where the trajectories progressively diverge. The normal reference anchors the original structure, paired prompts establish a contrast, and prompt refinement plus spatial biasing concentrate the difference on defect semantics and plausible object regions. This is an operational form of attribution, not a strictly identified causal explanation of real defects; its utility still requires evidence from synthesis and downstream detection. Core Idea: discover an editing region from normal-versus-anomalous denoising differences, then use that region to constrain later generation, jointly producing a local defect and its supervision mask.

Method

Overall Architecture

Inputs are a normal image, a normal prompt, an anomaly prompt differing by an added anomalous attribute, and a descriptive phrase that refines defect semantics. Outputs are an image containing a local defect and a final binary defect mask, not a direct anomaly decision for a real test image. The method performs "Prompt refinement," starts synchronized branches from a partially noised normal-image latent for "Spatially guided delta attribution," and finishes with "Mask-constrained late refinement." SAM supplies the early object-foreground prior, whereas the attribution-derived intermediate mask supplies the late prior; the foreground mask and defect mask have different roles.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    Prompt["Paired prompts and<br/>defect description"] --> Refine["Prompt refinement"]
    Image["Normal reference image"] --> Init["VAE encoding and partial noising<br/>SAM foreground mask"]
    Refine --> Delta["Spatially guided delta attribution"]
    Init --> Delta
    Delta -->|Intermediate mask; reset accumulator| Late["Mask-constrained late refinement"]
    Init -->|Normal reference latent at matching timestep| Late
    Late --> Output["Defect image and<br/>final defect mask"]

The branches share a pretrained diffusion model, random seed, and schedule; they are not separately trained normal and anomalous generators. The VAE encodes the normal image, and reverse denoising begins after partial noising, avoiding the need to reinvent a same-category object from pure noise. This initialization anchors geometry and illumination and shortens the executed reverse trajectory, but the procedure still requires more than a single network forward pass. The paper's training-free designation means no task-specific fine-tuning of generator weights, not an absence of optimization at inference time.

Key Designs

1. Prompt refinement: concentrate semantic differences on the defect

The normal prompt describes the object, and the anomaly prompt adds a defect attribute to the same description, such as a crack. Directly appending many details about shape, depth, and texture can change the contextual representation of the entire sentence and introduce changes outside the anomaly. The authors instead encode the detailed description separately as a semantic anchor for the anomaly token, rather than inserting it directly into the main prompt. Anomaly semantic distillation moves that token toward the anchor so that a word such as crack carries more specific morphology or severity information. A separate context constraint computes the mean representation of non-anomaly tokens in the anomaly prompt and uses mean-squared-error regularization to pull those tokens toward this context centroid. Importantly, the implementation described in the paper contracts toward the anomaly prompt's own centroid; it is not explicitly a token-by-token match to the normal prompt.

These objectives update only the anomaly prompt embedding through a few gradient iterations while keeping the diffusion model frozen. The first makes defect semantics more distinguishable, while the second attempts to suppress irrelevant contextual drift so that later latent differences better reflect anomalies. Neither guarantees strictly unchanged semantics for non-anomaly words, so subsequent spatial constraints remain necessary for background preservation. Some operators in the extracted anomaly-distillation and combined-loss equations are corrupted; this note explains the objectives supported by the prose without reconstructing an unverified exact loss. The paper describes a fixed step size and a small number of iterations, but the inspected main text does not provide a complete numerical configuration for reproduction.

2. Spatially guided delta attribution: localize defects through synchronized latent divergence

The defect location is unknown early in denoising, so a SAM foreground mask first identifies plausible surfaces that should respond to the defect description. Before the cross-attention softmax, the method adds a spatially mask-modulated bias only to the column corresponding to the anomaly token. It neither replaces the entire attention map with a mask nor strengthens all words; it encourages image features at foreground locations to attend to the defect word. Other tokens receive no direct logit bias, but softmax normalization can still change their resulting attention weights, so those weights are not mathematically fixed. The normal and anomaly branches then update their latents under the same schedule, measure channel-wise differences at each spatial position after the update, and accumulate them along the trajectory.

Following the textual definitions of Eqs. (8)-(9) in Section 3.2, the core operation is:

\[ d_{t-1}(u)=\left\|z_{t-1}^{(n)}(u)-z_{t-1}^{(a)}(u)\right\|_2, \qquad S_{t-1}(u)=S_t(u)+d_{t-1}(u). \]

Here \(u\) denotes a latent spatial position, the Euclidean norm is taken over channels, and \(S\) is a spatial accumulator initialized to zero. Although the abstract emphasizes differences in denoiser predictions, the explicit main-text operation compares latents after each reverse update rather than thresholding a raw noise-prediction map. Accumulation retains spatial divergence that persists across multiple steps, reducing dependence on one attention snapshot, although persistent irrelevant drift can also accumulate. At the intermediate stage, the accumulated map is lightly smoothed, normalized, and thresholded at 0.6 to obtain an intermediate binary mask. This mask specifies where later editing is allowed; it is not yet the final supervision mask supplied to the detector.

3. Mask-constrained late refinement: turn coarse localization into local synthesis and final labels

After extracting the intermediate mask, the accumulator is reset, and only late-stage latent differences are collected again to emphasize local detail and boundaries. The spatial prior for attention biasing switches from the SAM foreground to the intermediate mask, concentrating anomaly-token responses on the discovered defect region rather than the whole object. Meanwhile, latent inpainting mixes edited content inside the mask with normal-reference latents outside it, preventing anomaly semantics from spreading into the background. Reference latents come from noising the original normal-image encoding to the matching timestep, not from another normal image or a direct substitution of clean latents. Using the general notation of Eq. (5), the late-stage fusion in Section 3.2 is:

\[ z_{t-1}=M_{\mathrm{mid}}\odot\tilde z_{t-1} +(1-M_{\mathrm{mid}})\odot z^{\mathrm{src}}_{t-1}. \]

Here \(\tilde z_{t-1}\) is the proposed edited latent, \(z^{\mathrm{src}}_{t-1}\) is the normal reference at the matching noise level, and \(\odot\) denotes element-wise multiplication. Attention biasing thus encourages a defect within the region, whereas latent fusion limits continuing changes outside it; their roles are distinct. At the end of denoising, the VAE decodes the anomaly branch, and cleaning, normalization, and thresholding at 0.35 convert the late-stage accumulator into the final mask. The final threshold mainly affects annotation boundaries, while the intermediate threshold also changes the editable area and therefore can directly alter defect appearance. This yields a mask associated with the generation process, not a theoretical guarantee of exact agreement with genuine defect boundaries.

A Worked Example

Consider the pill-crack setting in Figure 6: the input is a normal pill image, the normal prompt describes the pill, and the anomaly prompt additionally requests a crack. A detailed description can emphasize larger, deeper cracks; prompt refinement transfers those details into the crack token without inserting the whole phrase into the main prompt. SAM first identifies the pill foreground; from a shared starting point, the normal branch tries to retain the pill while the anomaly branch attempts to form a crack under the spatial bias. Locations where the trajectories keep diverging brighten in the accumulator and become editable regions after intermediate thresholding. The crack is refined inside that region during late denoising, the matching-timestep normal reference maintains the outside region, and the method returns a cracked image and final mask. This is a walkthrough connecting the method to Figure 6, not an additional measured single-image success case, and no crack dimensions are invented.

Loss & Training

The default backbone is Stable Diffusion v1.5; the text reports \(T=100\), partial-noise ratio \(\gamma=0.3\), and intermediate/final thresholds of 0.6/0.35. However, Section 3.2 also defines \(t_{\mathrm{start}}=\gamma T\) and \(t_{\mathrm{mid}}=\lfloor T/2\rfloor\), which give 30 and 50 respectively. If both use the same decreasing timestep index, a trajectory beginning at 30 never passes through 50; the paper does not clearly distinguish full-schedule indices from executed-step indices. This note preserves the reproducibility ambiguity rather than silently redefining the midpoint as half the executed trajectory or presenting that interpretation as verified implementation behavior. The zero-shot constraint applies to anomaly synthesis: no real anomalous images or pixel labels and no generator fine-tuning, while prompt embeddings are still optimized at inference time. The downstream experiment separately trains a U-Net on 100 generated anomalous images and masks per category, all derived from the same normal reference for that category; real test data are used only for evaluation. Thus, a training-free generator must not be confused with an entirely training-free detection system.

Key Experimental Results

Main Results

The following table selects category means from Tables 2 and 3 on page 12; all values are on a percentage scale, with higher being better. AUROC-i and AP-i denote image-level AUROC and average precision; AUROC-p and AP-p are their pixel-level counterparts. The protocol generates 100 anomalies from the same normal image per category, uses the same U-Net architecture and training schedule, and evaluates real test splits without test-time adaptation.

Dataset	Method	AUROC-i	AP-i	AUROC-p	AP-p
MVTec AD	AnomalyAny	77.6	87.1	83.3	26.5
MVTec AD	AnoStyler	84.4	91.5	78.2	30.2
MVTec AD	DeltaDeno SD1.5	84.7	91.3	84.9	30.5
MVTec AD	DeltaDeno FLUX	89.7	94.8	85.4	41.5
VisA	AnomalyAny	74.7	77.9	87.6	7.5
VisA	AnoStyler	76.5	80.7	86.0	16.6
VisA	DeltaDeno SD1.5	82.1	85.7	90.1	11.6
VisA	DeltaDeno FLUX	85.1	87.6	90.2	15.0

Against AnomalyAny, the SD1.5 version improves MVTec AD image-level AUROC by 7.1 percentage points, but its gain over AnoStyler is only 0.3 percentage points. On VisA, SD1.5 achieves higher image-level AUROC but lower pixel-level AP than AnoStyler's 16.6, ruling out a claim of universal detection-metric superiority. The FLUX results demonstrate backbone scalability, but changing the backbone also changes pretrained capabilities and is not a controlled ablation of attribution alone.

The following generation averages come from Table 1 on page 11 for MVTec AD; both IS and IC-LPIPS are higher-is-better under the paper's convention. IC-LPIPS measures intra-cluster pairwise LPIPS distance as a diversity indicator; IS is a generation-quality proxy, not a certification of physically realistic industrial defects.

Method	IS	IC-LPIPS
Crop-Paste	1.51	0.14
DFMGAN	1.72	0.20
AnomalyDiff	1.80	0.32
AnomalyAny	2.02	0.33
AnoStyler	2.04	0.32
DeltaDeno	2.05	0.36

Mean IS exceeds AnoStyler by only 0.01, while IC-LPIPS exceeds AnomalyAny by 0.03; these support limited average gains, not superiority in every category. For example, Table 1 gives leather an IS of 1.77, below AnoStyler's 2.94, demonstrating substantial category variation. This table mixes few-shot and zero-shot methods with different supervision requirements and should not be treated as a ranking under identical input conditions.

Ablation Study

The following table comes from Table 4 on page 14 and reports downstream MVTec AD category means on a percentage scale, with higher being better. Variants generate the same number of anomalies from the same normal guidance image before U-Net training; the full configuration matches the SD1.5 row in Table 2.

Config	AUROC-i	AP-i	AUROC-p	AP-p
Without latent-mask inpainting A	79.7	87.6	81.2	25.8
Without attention biasing B	79.5	88.2	83.4	28.3
Without prompt refinement C	78.8	86.3	83.1	28.1
Full model	84.7	91.3	84.9	30.5

Key Findings

• Removing prompt refinement reduces image-level AUROC from 84.7 to 78.8, a 5.9-percentage-point drop, the largest drop in this metric among the reported module removals.
• Removing latent-mask inpainting reduces pixel-level AP from 30.5 to 25.8, a 4.7-percentage-point drop, highlighting the importance of background and editing-boundary constraints for localization supervision.
• Figure 5 provides primarily qualitative threshold analysis: the intermediate threshold affects the generated region, whereas the final threshold mostly changes the mask; no unreported optimal metric should be inferred.
• In Table 2, FLUX reaches 100.0 image-level AUROC on screw but only 46.6/0.2 pixel-level AUROC/AP, illustrating a severe separation between detecting an anomaly and locating it accurately.

Highlights & Insights

• Masks come from differences across a multistep generation trajectory rather than an additionally trained defect segmenter. Localization and synthesis share internal state, naturally associating generated images with candidate supervision regions.
• The two masks serve distinct purposes: the intermediate mask controls editing, and the final mask provides labels. Separating the control region from the annotation boundary avoids forcing one threshold to serve two different objectives.
• Prompt refinement is not merely prompt expansion: it strengthens anomaly tokens and reduces unrelated contextual drift. Making conditional differences easier to interpret may be more helpful for delta localization than simply increasing textual detail.

Limitations & Future Work

• The authors acknowledge limitations in current zero-shot anomaly-generation evaluation protocols and suggest future multimodal large language model assistance, but this paper does not validate the reliability of such replacement evaluation.
• Semantic distillation offers limited fine-grained defect control; generated anomalies mostly involve structural or surface changes, while abstract logical anomalies remain difficult.
• Synthetic and real defect-type distributions differ; VisA pixel-level AP and screw localization show that a stronger backbone does not automatically eliminate this gap.
• For reproduction, conflicting timestep definitions, corrupted extracted equations, and insufficient prompt-optimization configuration require clarification from original equations or implementation.
• From a reader's perspective, dual branches, SAM, and prompt optimization all incur computation; the inspected main text lacks a unified runtime comparison sufficient to assess deployment throughput.
• The code link comes from the paper's public-release plan and was not checked online here; attribution of generated regions should not be treated as a causal explanation of real industrial faults.

Related Work & Insights

• Compared with AnomalyAny: both address zero-shot anomaly generation, but DeltaDeno replaces coarse cross-attention localization with spatial differences between synchronized denoising trajectories and uses late inpainting to constrain spill-over. Benefits must be assessed by category, not just mean scores.
• Compared with DiffEdit: DiffEdit already contrasts source and target prompts to obtain editing masks, so prompt differencing itself is not new. DeltaDeno focuses on multistep accumulation, prompt refinement, and a two-stage localization-and-inpainting combination for anomaly synthesis.
• Compared with AnomalyDiffusion: the latter fine-tunes using real defects and masks; DeltaDeno removes that supervision requirement but also lacks direct assurance of matching the true defect distribution.
• Possible extension: consistency between local generated changes and the final mask could filter synthetic samples before detector training, reducing incorrect supervision. This is a reader-proposed direction, not an experiment completed in the paper.

Rating

• Novelty: 4/5. Combining trajectory differences, condition refinement, and two-stage local editing contributes clearly to zero-shot anomaly generation, while retaining connections to prior differential editing.
• Experimental Thoroughness: 4/5. Generation metrics, two downstream datasets, different backbones, and module ablations are covered, but unified efficiency and statistical-variability evidence are missing.
• Writing Quality: 3/5. The overall procedure is clear, but timestep indexing is ambiguous and the current extraction does not permit complete verification of loss details.
• Value: 4/5. Useful for bootstrapping data when real anomalies are unavailable, subject to validating localization quality for the target category.