Skip to content

title: >- [Paper Note] Beyond the Black Box: Identifiable Interpretation and Control in Generative Models via Causal Minimality description: >- [ECCV2026][Image Generation][Causal Minimality] A theoretical framework for hierarchical selection models under causal minimality, proving component-wise identifiability of generative representations to enable hierarchical concept extraction and fine-grained multi-level steering. tags: - ECCV2026 - Image Generation - Causal Inference - Interpretability - Diffusion Models - Sparse Autoencoders date: 2026-09-19 content_hash: f50a96ce2572eff3

Beyond the Black Box: Identifiable Interpretation and Control in Generative Models via Causal Minimality

Conference: ECCV 2026
Paper: ECCV 2026
Area: Image Generation
Keywords: Causal Minimality, Hierarchical Selection Models, Component-wise Identifiability, Sparse Autoencoders, Multi-level Concept Steering

TL;DR

This paper introduces a theoretical framework for hierarchical selection models grounded in the causal minimality principle, mathematically establishing component-wise identifiability of latent representations in generative models under sparsity constraints and demonstrating grounded hierarchical concept graph extraction, multi-level editing, and state-of-the-art concept unlearning on text-to-image diffusion models.

Background & Motivation

Modern deep generative models, particularly text-to-image diffusion models such as Stable Diffusion and Flux, have demonstrated remarkable capabilities in synthesizing photorealistic and visually intricate imagery. However, their internal representations have grown increasingly complex and opaque, operating effectively as black boxes. This pervasive opacity impedes our ability to understand internal knowledge organization, severely restricts our capacity to exert reliable and targeted control over generation behavior, and complicates the critical task of aligning these models with human intentions and safety standards.

Recent empirical approaches have employed sparse autoencoders (SAEs) to probe intermediate feature representations in diffusion models, uncovering interpretable directions and semantic specializations. Nevertheless, these heuristic methods largely lack rigorous theoretical foundations; interpretations extracted without mathematical guarantees risk being subjective, entangled, or biased, rendering them untrustworthy for risk-sensitive deployments. Crucially, conventional hierarchical causal models treat higher-level concepts as confounders that causally influence lower-level features while assuming that lower-level features remain conditionally independent given higher-level parents. In real-world visual generation, however, an abstract concept (such as a "bicycle") is defined by the coordinated, coherent configuration of its lower-level constituent parts (wheels, frame, handlebars). Modeling these complex intra-level dependencies within traditional frameworks requires introducing dense intra-layer causal edges, yielding prohibitively complex graphs that defy tractable identification.

This paper addresses this fundamental challenge by invoking the principle of causal minimality—the foundational premise that among all causal models capable of explaining the observed data distribution, the simplest is preferred. Rather than treating higher-level concepts as confounders, the authors formalize high-level concepts as selection variables (colliders) determined by the constrained composition of lower-level variables. Core idea: formulate generative representations via hierarchical selection models, prove that causal minimality induces component-wise identifiability under graph sparsity constraints, and operationalize these constraints through timestep-specific sparse autoencoders and causal discovery to extract innate hierarchical concept graphs for fine-grained multi-level steering.

Method

Overall Architecture

The framework establishes a closed-loop pipeline spanning theoretical identifiability guarantees, diffusion timestep mapping, sparse concept extraction, cross-level causal discovery, and downstream model intervention. Theoretically, text prompts \(D\), multi-level visual concepts \(Z = [Z_1, \dots, Z_{L_V}]\), and generated images \(X\) are conceptualized as a hierarchical selection system where higher-level variables act as selection variables over their constituent lower-level parents. Under smoothness, variability, and minimal sparsity conditions, component-wise identifiability is formally established. Empirically, the iterative denoising trajectory of diffusion models naturally aligns with this hierarchy: high-noise timesteps (\(t=899\)) govern coarse structural layouts, while low-noise timesteps (\(t=100\)) control fine-grained local textures. By training \(K\)-sparse SAEs at distinct timesteps and applying the PC causal discovery algorithm across sparse features, an explicit directed acyclic concept graph is extracted and leveraged for localized, multi-level feature steering.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Input text prompt D and noisy latents X_t"] --> B["Timestep-to-hierarchy mapping<br/>t=899 high-level / t=500 mid-level / t=100 low-level"]
    B --> C["Hierarchical selection model formulation<br/>High-level concepts act as selection colliders"]
    C --> D["Minimality-driven sparse feature learning<br/>K-sparse SAE extracts atomic concepts"]
    D --> E["Cross-level causal discovery<br/>PC algorithm builds hierarchical concept graph"]
    E --> F["Fine-grained model steering & control<br/>Targeted unlearning and multi-level editing"]

Key Designs

1. Hierarchical Selection Models: High-Level Concepts as Colliders of Constituent Components

To resolve the limitation of traditional hierarchical models—which fail to capture coordination among lower-level components without introducing dense horizontal edges—the authors formulate the data-generating mechanism via a selection mechanism. In this hierarchy, concepts at level \(l\) are denoted by continuous variables \(Z_l \in \mathbb{R}^{n(Z_l)}\), where smaller \(l\) corresponds to higher abstraction (\(l=1\) most abstract, \(l=L_V\) most concrete). Higher-level concepts \(V_l\) emerge as deterministic functions of their lower-level constituents \(V_{l+1}\): $\(V_l := g_{V_l}(V_{l+1})\)$ Because observations in natural data distributions are conditioned on meeting structural coherence criteria (i.e., \(V_l\) taking values in a valid subset \(\Omega\)), conditioning on the collider \(V_l\) induces natural statistical dependencies between components in \(V_{l+1}\) (e.g., \(V_{l+1,i} \to V_l \leftarrow V_{l+1,j}\)). The generative sampling path reverses this structural formulation from top to bottom: $\(Z_0 \sim P(Z_0), \quad Z_l \sim P(Z_l \mid Z_{l-1}), \quad l \in \{1, \dots, L_V + 1\}\)$ where \(Z_0 := D\) denotes discrete text conditioning and \(Z_{L_V+1} := X\) represents the observed continuous image. This selection structure captures complex part-whole coordination with minimal graphical edges.

2. Component-Wise Identifiability under Causal Minimality

Existing non-linear latent variable identifiability proofs typically guarantee only subspace-level recovery, leaving individual concepts entangled. This work introduces four formal conditions for visual concept identification (Condition 1): - Informativeness: There exists a diffeomorphism \(g_l: (Z_l, \epsilon_l) \mapsto X\) relating latent levels to observed images; - Smooth Density: The transition density \(p(z_{l+1} \mid z_l)\) is smooth across all levels; - Sufficient Variability: For any concept and its parent set \(\tilde{Z} := \text{Pa}(Z)\), score function vectors exhibit linear independence across distinct configurations; - Sparse Connectivity (Minimality): For each parent concept \(\tilde{Z}\), there exists a subset of children \(Z \subseteq \text{Ch}(\tilde{Z})\) whose unique common parent is \(\tilde{Z}\), namely \(\bigcap_{Z \in \mathcal{Z}} \text{Pa}(Z) = \{\tilde{Z}\}\).

Theorem 2 proves that if a true data-generating process satisfies Condition 1, and an alternative estimated model \(\hat{\theta}_V\) matches the observed marginal distribution \(P(X)\) while satisfying the minimality constraint on graph parent cardinality: $\(n(\text{Pa}(\hat{Z})) \le n(\text{Pa}(Z))\)$ then the learned latent variables \(\hat{Z}_l\) achieve component-wise identifiability for every hierarchical level \(l \in [L_V]\). That is, there exists a permutation \(\pi\) and invertible functions \(h_i\) such that \(\hat{Z}_i = h_i(Z_{\pi(i)})\), guaranteeing that each learned representation component isolates an authentic, unmixed semantic concept.

3. Timestep-Specific Concept Extraction and Targeted Hierarchical Steering

To operationalize the theoretical sparsity constraint in continuous diffusion models, the authors introduce a two-step identification protocol: 1. Level-Specific Concept Learning: Separate \(K\)-sparse SAEs are trained on intermediate U-Net / DiT features at discrete diffusion timesteps (specifically \(t=899\), \(t=500\), and \(t=100\)). Enforcing exact top-\(K\) sparsity enforces the parent-cardinality minimality constraint required by Theorem 2; 2. Cross-Level Causal Graph Discovery: The PC causal discovery algorithm is executed across activations from different timesteps to recover the explicit directed graph structure relating abstract concepts (e.g., cat face) to intermediate regions (upper face, forehead) and localized details (eyes, mouth).

For feature steering, an intermediate feature \(x\) is encoded into latents \(z = E(x)\), modified along the extracted steering direction \(v\) as \(x' = x + \lambda D(v)\), and reinjected into the diffusion backbone strictly at its corresponding timestep. Restricting intervention to the designated temporal level prevents unwanted global distortions and structural hallucinations common to monolithic steering approaches.

Key Experimental Results

Main Results

1. Model Unlearning Benchmark Comparison The method was evaluated on four concept erasure benchmarks targeting nudity suppression (I2P, RING-A-BELL across 3 splits, P4D, and UnlearnDiffATK) alongside generation fidelity on 10K MS-COCO prompts (Table 1 in original paper).

Method I2P ↓ RING-A-BELL (AVG) ↓ P4D ↓ UATK ↓ COCO FID ↓ COCO CLIP ↑
SD 1.4 (Baseline) 17.80 88.10 98.70 69.70 16.71 31.30
ESD 2.87 28.42 15.49 2.87 18.18 30.20
MACE 1.51 0.70 2.82 1.51 16.80 28.70
UCE 0.87 10.87 9.86 0.87 17.99 30.20
RECE 0.72 4.91 5.63 0.72 17.74 30.20
ConceptSteer 0.36 7.02 1.99 2.11 18.67 30.80
Ours 0.26 1.17 0.66 2.11 17.02 31.30

Note: The proposed method attains an I2P rate of 0.26 (±0.06) and reduces P4D unsafe generation to 0.66%, while maintaining a low FID of 17.02 and high CLIP score of 31.30 on COCO, proving that targeted unlearning does not impair general visual fidelity.

2. Spatial Activation Spread and Deactivation Sensitivity across Noise Levels To validate the temporal hierarchy hypothesis, the authors quantified the spatial coverage of top-\(k\) SAE features and measured perceptual/semantic divergence (L1, LPIPS, CLIP, DINO) upon deactivating the top-1 feature at specific timesteps (Table 2 in original paper).

Timestep \(t\) Top1 Area Top3 Area Top10 Area Deact. L1 ↑ Deact. LPIPS ↑ Deact. DINO ↓
\(t=100\) (Fine Details) 0.27 0.21 0.15 0.004 0.002 0.999
\(t=500\) (Mid-Level Structure) 0.30 0.25 0.17 0.013 0.020 0.993
\(t=899\) (High-Level Semantics) 0.53 0.41 0.24 0.070 0.220 0.903

Note: Activations at \(t=899\) cover over 53% of spatial area and top-1 deactivation causes an LPIPS shift of 0.220, whereas deactivations at \(t=100\) alter LPIPS by only 0.002, empirically confirming that high noise steps govern global semantics while low noise steps encode local details.

3. Controllable Image Generation Fidelity and Alignment On three editing tasks—adding tabby patterns to cats, adding mountains to landscapes, and replacing rocks with textured tree stumps—the method was evaluated against base SD 1.4 and non-hierarchical SAE steering (Table 3 in original paper).

Task & Metric SD 1.4 Baseline SD 1.4 (SAE w/o Hierarchy) SD 1.4 (Ours)
Add tabby pattern – CLIP-I (Image Preserv.) ↓ 0.91 ± 0.05 0.83 ± 0.07 0.93 ± 0.04
Add tabby pattern – CLIP-T (Text Align.) ↑ 0.27 ± 0.00 0.28 ± 0.02 0.28 ± 0.01
Add mountains – CLIP-I (Image Preserv.) ↓ 0.84 ± 0.06 0.83 ± 0.04 0.91 ± 0.03
Add mountains – CLIP-T (Text Align.) ↑ 0.33 ± 0.01 0.32 ± 0.01 0.33 ± 0.01
Replace rock w/ stump – CLIP-I (Image Preserv.) ↓ 0.93 ± 0.02 0.95 ± 0.02 0.96 ± 0.02
Replace rock w/ stump – CLIP-T (Text Align.) ↑ 0.31 ± 0.01 0.29 ± 0.01 0.31 ± 0.01

Ablation Study

The impact of sparsity was analyzed at timestep \(500\) by varying the active feature budget \(K \in \{100, 10, 4\}\) in the SAE (Figure 4 in original paper): - Insufficient Sparsity (\(K=100\)): Produces dense, interconnected graphs where high-level facial representations are tangled with localized eye and ear nodes, degrading semantic clarity. - Balanced Minimality (\(K=10\)): Recovers a cleanly organized hierarchical tree: high-level node 3556 ("Full face") branches into mid-level nodes 3044 ("Mid-face") and 1026 ("Forehead"), which further connect to localized nodes 3066 ("Eyes") and 762 ("Mouth"). - Excessive Sparsity (\(K=4\)): Over-prunes meaningful edges, disconnecting intermediate structures and failing to capture essential generative dependencies.

Key Findings

  • Simultaneous Multi-Level Editing: Disentangling levels allows decoupled compound edits. In drink generation, high-level node 2212 controls overall cup geometry, while mid-level node 3372 encodes ice cubes. Simultaneously steering node 2212 positively and node 3372 negatively produces a completely redesigned glass while cleanly removing ice without background degradation.
  • Temporal Localization Eliminates Artifacts: Applying negative steering vectors across all diffusion timesteps causes severe visual corruption (e.g., distorting human figures or wiping out entire backgrounds). Constraining the modification to the specific noise level matching the concept's abstraction depth eliminates unwanted collateral distortion.

Highlights & Insights

  • Theoretical Bridge from Causal Minimality to Representation Learning: This work provides the first formal proof that causal minimality—manifested as sparsity—guarantees component-wise identifiability in continuous hierarchical selection models, placing empirical SAE techniques on solid mathematical footing.
  • Harmonization with Diffusion Denoising Dynamics: The hierarchical causal graph naturally mirrors the coarse-to-fine trajectory of diffusion models, transforming heuristic observation of timestep semantics into actionable causal levers for precise steering.
  • Surgical Unlearning without Capability Degradation: By intervening directly on isolated causal nodes at designated timesteps, the method achieves superior concept suppression (I2P error down to 0.26%) while maintaining zero drop in general image fidelity (COCO FID 17.02).

Limitations & Future Work

  • Regularity and Variability Assumptions: The mathematical proofs require sufficient variability and smoothness across latent distributions. In scenarios characterized by severe long-tail data scarcity or absolute concept co-occurrence, separate concept identifiability remains challenging.
  • Discrete Timestep Approximation: The current implementation selects three discrete timesteps (899, 500, 100) to represent the hierarchy. Formulating continuous-time causal flows along probability flow ODE trajectories represents an important future direction.
  • Scaling to Large-Scale Video and Multimodal Diffusion: Extending hierarchical selection discovery to spatial-temporal video architectures (e.g., Sora, Wan) could reveal dynamic temporal concept graphs across motion and appearance latents.
  • vs Traditional Hierarchical Causal Discovery (Latent Tree Models / ICA): Previous frameworks model latents as confounders and struggle with intra-layer dependencies or achieve only subspace identifiability. This paper introduces selection models with collider latents, elegantly preserving part-whole dependencies while proving component-wise identifiability.
  • vs Empirical Diffusion SAEs (ConceptSteer / Revelio): Prior studies use SAEs as heuristic post-hoc probes without identifiability guarantees. This work demonstrates that sparsity is an instantiation of causal minimality and uses causal discovery to organize latent features into an operational hierarchy for multi-level control.

Rating

  • Novelty: ⭐⭐⭐⭐⭐ Formulates hierarchical selection models with causal minimality and proves component-wise identifiability.
  • Experimental Thoroughness: ⭐⭐⭐⭐⭐ Comprehensive evaluation across concept extraction, spatial spread metrics, multi-level editing, and four concept unlearning benchmarks.
  • Writing Quality: ⭐⭐⭐⭐⭐ Clear theoretical exposition bridging abstract causal principles with practical diffusion mechanics.
  • Value: ⭐⭐⭐⭐⭐ Provides a foundational causal paradigm for interpretable, controllable, and safely aligned deep generative models.