LACON: Training Text-to-Image Model from Uncurated Data

Conference: ECCV 2026
Paper: ECCV Official
Project: https://zhiyangliang.github.io/LACON
Area: Image Generation
Keywords: text-to-image generation, uncurated data, explicit quality conditioning, diffusion model, controllable generation

TL;DR

LACON overturns the prevailing filter-first paradigm by repurposing five core quality signals (aesthetic score, watermark probability, clarity, entropy, and luminance) into explicit continuous conditioning vectors via Gaussian-weighted cluster centroids, training on 100% raw uncurated data to delineate quality boundaries and achieve artifact-free, high-fidelity controllable generation.

Background & Motivation

The recent revolutionary leaps in large-scale text-to-image (T2I) generation are heavily anchored on massive web-scale corpora comprising billions of image-text pairs. However, raw internet data routinely suffers from low visual fidelity, degraded aesthetics, intrusive watermarks, severe blur, and poor semantic alignment. To shield models from degradation, prevailing state-of-the-art systems overwhelmingly adhere to an aggressive filter-first curation paradigm. For instance, Stable Diffusion relies strictly on the aesthetic subset LAION-Aesthetics, Qwen-Image incorporates a complex seven-stage sequential filtering cascade, and Hunyuan-Image implements a multi-stage pruning pipeline that discards more than 55% of the initial raw data.

Despite its ubiquity, this filter-first paradigm introduces two fundamental flaws. First, it is conceptually wasteful and statistically inefficient: pruning more than half of the corpus arbitrarily purges valuable long-tail concepts, diverse real-world geometric structures, and rich contextual knowledge that rarely appear in polished subsets. Second, and more critically, it induces an internal knowledge blind spot within the model. By exposing the network exclusively to a narrowly curated slice of pristine imagery, the model acquires a strong prior for "high quality" but develops zero explicit comprehension of "low quality". Consequently, it fails to delineate the explicit decision boundaries separating good and bad distributions. At inference time, practitioners are forced into heuristic guesswork—tuning negative prompts with classifier-free guidance (CFG) to push generations away from failure modes the model never explicitly encountered during pretraining.

The angle of attack in this work is straightforward yet profound: rather than treating quality metrics as destructive binary gates for sample removal, re-purpose them as constructive, learnable conditioning signals. Core idea: transform multi-dimensional quality signals—such as aesthetic score, watermark probability, clarity, entropy, and luminance—into explicit quantitative continuous condition labels, map them into smooth soft embeddings via Gaussian-weighted Cluster Centroids (GCC) concatenated with noisy latents, and train generative models over 100% of raw uncurated data to acquire complete quality-spectrum awareness and enable fine-grained inference-time controllability.

Method

Overall Architecture

LACON (Labeling-and-Conditioning) reformulates the text-to-image objective by shifting from the standard conditional distribution \(p(x|y)\) trained on a filtered subset to a comprehensive joint distribution \(p(x|y, s)\) trained over the entire uncurated distribution, where \(x\) denotes the visual image, \(y\) represents the text prompt, and \(s\) is a multi-dimensional continuous attribute vector capturing fine-grained quality dimensions.

The end-to-end framework comprises three primary stages. In the offline annotation phase, an automated labeling pipeline processes the entire 110M raw dataset, computing five quantitative continuous quality scalars for each sample. During training, a Gaussian-weighted Cluster Centroid (GCC) embedding module converts continuous scalar scores into soft condition embeddings matched to the noisy latent channel dimension, followed by context conditioning that directly prepends these tokens to the latent patch sequence for the DiT backbone. Finally, at inference time, flexible guidance strategies—either fixing high-quality condition targets (LACON-S) or performing decoupled multi-attribute CFG scaling (LACON-A)—steer the sampling trajectory along desired quality dimensions while eliminating blank boundary artifacts and watermarks.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Raw Uncurated Image-Text Data<br/>100% retained with long-tail world knowledge"] --> B["Data Attribute Labeling<br/>Extract aesthetics, watermark, clarity, entropy, luma"]
    B --> C["Gaussian-weighted Cluster Centroids (GCC)<br/>RBF-anchored soft interpolation into continuous tokens"]
    C --> D["Context Conditioning & Full-Spectrum Training<br/>Latent-token concatenation + Flow Matching objective"]
    D --> E["Decoupled Guidance Inference<br/>LACON-S fixed high-quality / LACON-A per-attribute CFG"]
    E --> F["High-Fidelity Controllable Image Generation<br/>Suppressed low-entropy artifacts & preserved long-tail concepts"]

Key Designs

1. Data Attribute Labeling: Repurposing Filtering Criteria into Quantitative Condition Vectors To capture both high-level semantic aesthetics and low-level physical characteristics, LACON extracts five quality metrics commonly utilized in conventional pruning pipelines, organizing them into a five-dimensional conditioning vector \(s = [s_{\text{aes}}, s_{\text{wat}}, s_{\text{cla}}, s_{\text{ent}}, s_{\text{luma}}]\). Specifically, the aesthetic score \(s_{\text{aes}} \in [0, 10]\) assesses perceived visual appeal; the watermark probability \(s_{\text{wat}} \in [0, 1]\) detects artificial logos and text overlays; clarity \(s_{\text{cla}}\) is measured via Laplacian variance reflecting high-frequency image sharpness, clipped at 3000 to suppress extreme outliers; Shannon entropy \(s_{\text{ent}} \in [0, 8]\) gauges local information density and texture complexity; and luminance \(s_{\text{luma}} \in [0, 1]\) captures visual brightness in the HSV color space. Labeling every uncurated sample with these explicit coordinates ensures zero data discarding while establishing an exhaustive quality coordinate system.

2. Gaussian-weighted Cluster Centroids (GCC): Smooth Soft Embedding of Continuous Scalars Projecting continuous scalar scores into learnable representations that a diffusion transformer can digest requires expressiveness and numerical stability. Standard linear interpolation with two boundary tokens lacks capacity, discrete binning causes sharp step discontinuities across bin thresholds, and Fourier-feature embeddings suffer from high-frequency oscillations and hyper-parameter sensitivity. To overcome these limitations, LACON introduces Gaussian-weighted Cluster Centroids. For the \(k\)-th quality dimension, \(N\) fixed, non-learnable scalar anchors \(p^{(k)} = \{p_1^{(k)}, \dots, p_N^{(k)}\}\) are linearly distributed across the expected metric range, and each anchor is paired with a learnable centroid embedding vector \(c_i^{(k)} \in \mathbb{R}^d\) matching the latent channel depth. Given a sample score \(s_k\), affinity scores \(u_i^{(k)}\) across all anchors are computed using a Gaussian Radial Basis Function (RBF) kernel: $\(u_i^{(k)} = \exp\left(-\frac{(s_k - p_i^{(k)})^2}{2\sigma_k^2}\right)\)$ where the bandwidth parameter \(\sigma_k\) is set strictly to half the distance between adjacent centroid anchors, providing an adaptive, smooth overlap tailored to each attribute's scale. Normalizing these affinities yields weights \(w_i^{(k)} = u_i^{(k)} / \sum_{j=1}^N u_j^{(k)}\), and the final attribute condition embedding \(e_k\) is synthesized as a convex combination of learnable centroids: $\(e_k = \sum_{i=1}^N w_i^{(k)} c_i^{(k)}\)$ This soft assignment retains the compact parameter footprint of discrete binning while providing smooth differentiability and expressive modeling of local score geometry across diverse data qualities.

3. Context Conditioning & Full-Spectrum Training: Early Multi-Layer Quality Awareness Rather than injecting quality attributes through separate cross-attention layers—which incurs computational overhead and risks having quality tokens overshadowed by dense text prompts—LACON adopts a clean context conditioning strategy. The individual condition tokens are concatenated into an attribute sequence \(e = [e_1, \dots, e_5]\) and prepended directly to the visual latent tokens \(x_t\) along the sequence dimension as the primary input to the DiT backbone. Text embeddings \(y\) continue to interface via standard cross-attention. This early sequence-level fusion forces the transformer to process quality information starting from the initial self-attention layers, allowing the network to internalize explicit quality manifolds and separate degraded artifacts from semantic content without sacrificing generation fidelity.

4. Flexible Guidance Inference: Quantitative Quality Control via LACON-S and LACON-A At inference time, explicit conditioning uncouples quality control from heuristic prompt engineering, enabling users to dictate output attributes deterministically. LACON introduces two complementary inference modes: - LACON-S (Standard Guidance): Engineered for maximal sampling efficiency, this scheme clamps the quality vector \(s\) to ideal target values (e.g., maximum aesthetic score, zero watermark probability, elevated clarity and entropy, balanced luminance) and applies standard classifier-free guidance exclusively to the text prompt \(y\). This configuration incurs zero additional neural network evaluations beyond standard CFG. - LACON-A (Aggressive Multi-Condition Guidance): Designed for granular attribute control, this mode applies independent guidance scaling to the text condition and each of the \(K = 5\) quality attributes at each denoising step \(t\), executing \(K + 2\) parallel forward passes: $\(v_t^a = v_{\text{base}} + \omega_c (v_{\text{text}} - v_{\text{base}}) + \sum_{k=1}^K \omega_k (v_{s_k} - v_{\text{text}})\)$ where \(v_{\text{base}} = v_\theta(x_t, t, \emptyset, s_{\text{base}})\), \(v_{\text{text}} = v_\theta(x_t, t, y, s_{\text{base}})\), and \(v_{s_k}\) denotes the velocity prediction where only the \(k\)-th attribute is set to its high-quality target while other attributes remain at base levels. Users can dynamically amplify \(\omega_{\text{aes}}\) or \(\omega_{\text{cla}}\) to enhance artistic aesthetics and sharpness, or selectively suppress low-entropy artifacts like blank borders.

Loss & Training

The framework is optimized end-to-end using the Flow Matching objective along linear interpolation trajectories \(x_t = (1-t)x + t\epsilon\), where \(x \sim p(x)\) and \(\epsilon \sim \mathcal{N}(0, I)\). The model parameters \(\theta\) are trained to predict the vector field velocity \(v_\theta(x_t, t, s, y) = \epsilon - x\): $\(\mathcal{L}_{\text{FM}} = \mathbb{E}_{t, x, \epsilon, s, y} \left[ \| v_\theta(x_t, t, s, y) - (\epsilon - x) \|_2^2 \right]\)$ Experiments primarily utilize the DiT-based Sana backbone across Sana-0.6B and Sana-1.6B variants. The training corpus aggregates 110M images (100M from LAION-2B-EN and 10M from internal collections), with all captions standardized offline via GPT-4o to eliminate textual noise. Sana-1.6B is pretrained on \(512\times512\) images for 150K steps with a global batch size of 2048 across 32 NVIDIA A100 GPUs, followed by 20K additional fine-tuning steps at \(1024\times1024\) resolution.

Key Experimental Results

Main Results

Quantitative evaluations are conducted across GenEval (compositional text-to-image alignment), DPG (detailed prompt following), and FID on MJHQ-30K at both \(512\times512\) and \(1024\times1024\) resolutions. Baselines include: Baseline-A (strict filtering, retaining \(\approx 5\%\) of data), Baseline-B (optimal filtering threshold, retaining \(\approx 65\%\) of data), Baseline-C (100% full raw data, trained without quality conditioning), and Baseline-D (100% raw data with inference-time negative prompting).

Resolution & Architecture	Method	Data Retention	GenEval ↑	DPG ↑	FID (MJHQ-30K) ↓
\(512\times512\) (Sana-0.6B)	Baseline-A (Heavy Filter)	\(\approx 5\%\)	55.0	68.1	14.9
	Baseline-B (Optimal Filter)	\(\approx 65\%\)	62.8	70.4	13.4
	Baseline-C (Full Uncurated)	100%	60.8	69.7	15.4
	Baseline-D (+ Negative Prompt)	100%	61.0	69.2	15.3
	LACON-S (Ours)	100%	64.4	71.9	11.9
	LACON-A (Ours)	100%	65.6	71.8	11.8
\(512\times512\) (Sana-1.6B)	Baseline-A (Heavy Filter)	\(\approx 5\%\)	58.3	73.4	14.0
	Baseline-B (Optimal Filter)	\(\approx 65\%\)	68.0	76.1	13.1
	Baseline-C (Full Uncurated)	100%	67.0	75.4	13.5
	Baseline-D (+ Negative Prompt)	100%	67.1	75.5	13.5
	LACON-S (Ours)	100%	70.9	77.3	12.0
	LACON-A (Ours)	100%	71.6	78.1	11.2
\(1024\times1024\) (Sana-1.6B)	Baseline-B (Optimal Filter)	\(\approx 65\%\)	68.3	76.3	12.9
	LACON-S (Ours)	100%	70.9	77.6	11.4
	LACON-A (Ours)	100%	71.5	78.8	11.3

To confirm generalization across generative paradigms, LACON was additionally evaluated on autoregressive models (Qwen3-0.6B and Qwen3-1.7B paired with LlamaGen VQ-VAE). LACON-A achieved 61.3 / 75.4 / 11.2 on Qwen3-0.6B and 70.3 / 80.1 / 10.9 on Qwen3-1.7B, substantially outperforming the strongest filtered Baseline-B (58.9 / 73.1 / 12.4 and 66.4 / 78.2 / 12.1 respectively).

Ablation Study

1. Continuous Token Injection Strategies (Evaluated on Sana-1.6B at \(512\times512\))

Injection Strategy	Mechanism Description	GenEval ↑	DPG ↑	FID ↓
Linear Interpolation	Two learnable boundary tokens per metric with scalar interpolation	69.5	76.1	12.7
Discrete Binning	Quantized score intervals assigned to discrete learnable tokens	69.8	77.2	12.4
Fourier Feature	Sinusoidal frequency encoding mapped through an MLP projection	70.4	77.0	12.5
GCC (Ours)	Fixed centroid anchors with Gaussian RBF soft assignment	71.6	78.1	11.2

2. Inference-Time Quality Signal Ablation (LACON-S on Sana-1.6B with individual signals removed)

Guidance Configuration	Removed Signal	GenEval ↑	DPG ↑	FID ↓	Sensitivity & Impact
LACON-S (Full)	None (All 5 signals active)	70.9	77.3	12.0	Baseline reference
w/o \(s_{\text{aes}}\)	Remove aesthetic condition	69.4	76.6	12.6	Largest degradation (-1.5 GenEval, +0.6 FID)
w/o \(s_{\text{cla}}\)	Remove clarity condition	69.7	76.8	12.5	Perceptual sharpness drops (+0.5 FID)
w/o \(s_{\text{ent}}\)	Remove entropy condition	69.8	77.0	12.2	Frequent blank border failure modes emerge
w/o \(s_{\text{wat}}\)	Remove watermark condition	70.2	77.2	12.1	Residual watermark artifacts faintly reappear
w/o \(s_{\text{luma}}\)	Remove luminance condition	70.4	77.3	12.1	Slight drop in exposure consistency

Key Findings

• Breaking the Quality-Quantity Tension: In conventional pipelines, scaling data while adjusting filtering thresholds inevitably encounters a bottleneck: strict filtering starves the model of diversity, whereas relaxing thresholds introduces toxic low-quality noise. LACON resolves this trade-off completely, leveraging 100% of raw data while outperforming the optimal 65% filtered subset (FID improved from 13.1 to 11.2 on Sana-1.6B).
• Closing the World Knowledge Gap: Visual inspections confirm that Baseline-B loses crucial long-tail concepts—such as Morpho butterfly, mineral terraces, and pastel elderly portraits—due to aggressive pre-filtering. LACON preserves full conceptual coverage from the uncurated corpus while keeping generated quality pristine.
• Aesthetic and Clarity Dominance: Conditioning on \(s_{\text{aes}}\) and \(s_{\text{cla}}\) drives the largest quantitative gains in fidelity and prompt alignment, while entropy conditioning (\(s_{\text{ent}}\)) specifically eliminates the blank boundary collapse characteristic of models trained on raw uncurated web images.

Highlights & Insights

• From Destructive Pruning to Constructive Conditioning: Instead of discarding billions of compute-heavy quality assessment outputs (aesthetic rankers, watermark detectors), LACON recycles these scores into explicit conditioning labels, transforming data curation into an additive supervision asset.
• Geometric Grace of Gaussian Cluster Centroids: GCC combines the compactness of discrete token tables with the mathematical smoothness of continuous kernels, sidestepping the boundary steps of binning and the instability of Fourier features.
• Replacing Heuristic Guesswork with Explicit Quality Coordinates: Eliminates brittle reliance on negative prompt tuning during inference by equipping generative models with an innate, calibrated navigation system across the entire data quality spectrum.

Limitations & Future Work

• Hand-Crafted Quality Metric Taxonomy: Current implementation relies on five pre-selected signals (aesthetics, watermark, clarity, entropy, luminance). It does not explicitly account for higher-level structural failures such as anatomical distortions (e.g., deformed hands) or geometric perspective inconsistencies.
• Inference Compute for Multi-Condition Guidance: While LACON-S runs at zero additional compute cost, LACON-A requires \(K+2\) forward passes per denoising step, scaling inference latency roughly 7-fold. Future work should investigate multi-condition distillation or trajectory caching.
• Generalization to Video and 3D Generative Models: Extending explicit quality conditioning to spatio-temporal artifacts (motion jitter, temporal flickering in video) and multi-view 3D consistency represents an exciting and natural next horizon.

Related Work & Insights

• vs Filter-First Industrial Pipelines (Qwen-Image, Hunyuan-Image): Standard industrial models spend immense compute engineering multi-stage cascades that discard over 50% of raw corpora. LACON demonstrates that training across all raw data with explicit quality labels yields superior quality and conceptual coverage under identical compute budgets.
• vs Coherence-Aware Diffusion & MIRO: Prior works examined conditioning on text-image coherence or reinforcement learning reward models. LACON establishes the first systematic framework mapping full-spectrum low-level and high-level curation criteria into unified continuous condition embeddings.
• vs Negative Prompting: Negative prompts operate indirectly within uncurated text-latent spaces, frequently triggering semantic distortions or unintended concept erasure. LACON provides a mathematically grounded, dual training-time counterpart that delivers deterministic quality steering.

Rating

• Novelty: ⭐⭐⭐⭐⭐ Re-imagines the foundational data curation paradigm for generative models by replacing data discarding with explicit continuous multi-condition modeling.
• Experimental Thoroughness: ⭐⭐⭐⭐⭐ Comprehensive evaluation across 0.6B and 1.6B scales, diffusion and autoregressive architectures, 5% to 100% retention sweeps, granular quality signal ablations, and long-tail concept analyses.
• Writing Quality: ⭐⭐⭐⭐⭐ Lucid problem formulation, rigorous mathematical grounding, and compelling visual evidence.
• Value: ⭐⭐⭐⭐⭐ Offers a transformative and highly practical blueprint for industrial foundation model pretraining on uncurated web-scale datasets.