ECCV 2026 Image Generation trajectory-level generation semantic hierarchy one-step flow matching intermediate-state editing representation-space generation

Trajectory Forcing: Structure-First Generation with Controllable Semantic Trajectories¶

Conference: ECCV 2026
Paper: ECCV 2026 official page / Project page
Area: Image Generation
Keywords: trajectory-level generation, semantic hierarchy, one-step flow matching, intermediate-state editing, representation-space generation

TL;DR¶

Trajectory Forcing (TF) rewrites the opaque denoising trajectory of diffusion/flow models into an explicit semantic coarse-to-fine path (object/background → parts → subparts → detail), where every level is rendered into a viewable image by a shared RAE decoder and can be edited mid-generation before the model continues; each level costs a single network evaluation (L=4 NFE), reaching FID 1.98 / IS 261.6 on class-conditional ImageNet 256×256 after only 80 training epochs.

Background & Motivation¶

Diffusion and flow models already produce strong images, yet their controllability remains essentially endpoint-centric: users supply conditions (class, text, reference image) and receive a final image, with nothing in between visible. Intermediate states do exist — denoising necessarily passes through them — but they were never designed as objects for human understanding or intervention; they are computational by-products discarded once generation ends. This is the opposite of how humans make pictures: art training repeatedly insists on establishing the large shapes and colour blocks first and refusing to chase detail early, and the speed-painting example in Fig. 3 shows the same thing, with every intermediate study being something an artist can hold up, compare, and repaint. The paper starts from this contrast and asks whether the generative trajectory itself can be made readable, semantic, and editable.

Prior work has in fact exploited trajectory structure, but only as a means of optimizing the final image. Diffusion Forcing assigns independent noise levels to individual tokens so that various generation orders become permissible on sequential data; DeCo factorizes pixel diffusion along frequency components so the main network specializes in low-frequency semantics while a light decoder adds high frequencies; Latent Forcing runs self-supervised latent features and pixels on separate schedules so latent structure appears before pixel detail — and then throws those latent features away; Next Visual Granularity (NVG) clusters bottom-up in VQ-VAE latent space and autoregressively predicts tokens level by level. What they share is that intermediate states are either implicit or disposable. The core tension is that turning an intermediate state into a genuine user interface requires two properties that usually conflict: it must be semantic (a person recognizes at a glance whether this level is the object silhouette or a particular part) and it must be decodable (a person can actually render and look at it). Pixel space fails on the first, because appearance, layout, and identity are entangled; VQ-VAE fails on the second, because its hierarchy is dictated jointly by greedy binary splits (k=2) and by latent resolution, so its depth is pinned by a formula like 1+log2(16×16)=9 and the resulting stages carry no clear semantic grouping (the comparison in Fig. 2). Worse, once generation is split into L stages, multi-step denoising per stage blows inference up from T evaluations to L×T.

The paper's angle is to find a substrate where semantic geometry and decodability hold at the same time: in the feature space of a pretrained visual representation (DINOv2), semantic factors (object identity, part structure, spatial layout) are already well separated, and under the representation alignment hypothesis a plain unsupervised clustering reliably recovers object-part-subpart decompositions; meanwhile Representation Autoencoders (RAE) have already trained a decoder that reconstructs images from DINOv2 features, so any point in that space is visually decodable. Combine that with one-step flow matching per level (in the MeanFlow/iMF line) rather than multi-step denoising, and the number of hierarchy levels becomes fully decoupled from the number of denoising steps. Core idea: treat the generative trajectory as a first-class object — use an unsupervised clustering hierarchy as the teacher, train a single cross-level-shared one-step flow model conditioned on the previous level's output, and make every intermediate state semantic, decodable, and editable, while turning "edit scope" and "cross-level structural consistency" into quantifiable metrics.

Method¶

Overall Architecture¶

The problem TF solves can be stated in one sentence: make the "noise → image" path grow a staircase of semantic steps that a human can read and intervene in. During training, an input image is encoded into dense DINOv2 features, unsupervised hierarchical clustering produces an "object/background → parts → subparts" teacher hierarchy, and every token is replaced by the mean feature of its assigned region, yielding a set of level canvases that serve as per-level denoising targets. A single DiT, shared across all levels, is then trained as a one-step flow model: each step samples a level l uniformly, takes the previous-level canvas \(z^{(l-1)}\) as a spatial condition and the level index l as a global condition, and predicts the clean canvas of that level in one shot. At inference the direction reverses: generation starts from pure noise at l=0 and proceeds down to the finest level l=L−1 (the original feature resolution), one network evaluation per level, L=4 in total. Every level's output can be decoded into an image immediately by the RAE decoder, and if a user modifies one level, only the levels after it are regenerated. Compared with progressive generators such as NVG, TF cuts inference from 184 NFE to 4 NFE and requires no custom multi-granularity VQ-VAE.

A note on the word "forcing": ⚠️ the paper never defines it explicitly, so the following is my reading based on context and citation lineage. One sense comes from the Diffusion Forcing / Latent Forcing line — those methods assign different noise levels to tokens to permit a certain generation order, whereas TF forces the order to be the semantic coarse-to-fine one and gives the model no freedom to choose. The other sense lives on the training side: the structural loss explicitly forces each level's prediction toward the region means the teacher specifies, as if the intermediate state were being guided along by the hand (in the teacher-forcing sense). This also explains why a single step suffices per level — the target has been constrained to be simple enough.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["input image → dense DINOv2 features"] --> B["teacher hierarchy and level canvases"]
    B -->|"region-mean targets"| C["hierarchy-conditioned one-step flow matching"]
    B -->|"structural supervision"| D["structural loss"]
    C --> D
    D -->|"trained model"| E["hierarchical sampling and trajectory-level editing<br/>L=4 NFE, edit level l* and regenerate downstream only"]
    E --> F["trajectory-aware metrics<br/>LIS and PMR"]
    E --> G["RAE-decoded output"]

Key Designs¶

1. Teacher hierarchy and level canvases: define what an intermediate state should look like, with zero manual annotation

If intermediate states are to be semantic, one must first answer "which level corresponds to which semantics." The paper's answer is to let the data speak: run unsupervised clustering on dense DINOv2 token features to obtain a fixed-depth hierarchy shared by all samples, giving each token a triple of cluster indices (object/background, part, subpart). The coarsest level uses nothing but K-means with K=2, after which a plain centre prior designates the cluster whose tokens are on average closer to the image centre as the object and the other as background — a foundation for the finer decomposition. Parts and subparts are then clustered with agglomerative clustering over the object tokens only, using a distance that glues semantics to space:

\[\mathcal{D}(i,j)=\cos\!\left\langle \mathbf{z}_i,\mathbf{z}_j\right\rangle+\alpha\left\lVert \mathbf{p}_i-\mathbf{p}_j\right\rVert\]

where \(\mathbf{p}_i\) is the normalized spatial coordinate of token \(i\) and \(\alpha\) controls the strength of the spatial regularizer. The point of that extra term is to keep semantically similar tokens that sit on opposite sides of the image from being merged, so each cluster stays spatially contiguous. Cutting the resulting dendrogram at two preset distance thresholds yields the two granularities: the higher threshold (0.65) produces finer subparts, the lower one (0.35) coarser parts.

Given the hierarchy, each level's denoising target is constructed by replacing every token with the mean feature of its region, except at the finest level, which keeps the original features:

\[z^{(l)}_i=\boldsymbol{\mu}^{(l)}_{R^{(l)}_i},\qquad \boldsymbol{\mu}^{(l)}_k=\frac{1}{|R^{(l)}_k|}\sum_{j\in R^{(l)}_k}\mathbf{z}_j\]

The resulting canvases form a semantic trajectory from piecewise-constant to full-fidelity latent: l=0 is a two-colour block map of object versus background, l=1 brings in parts, l=2 subparts, and l=3 returns to real detail. This is exactly what "structure-first" means here — not a separate structure branch followed by an appearance branch, but a gradual de-quantization inside one DINOv2 space: the coarse region partition (who is where) is fixed first, and the per-token variation inside regions is left to the finer levels. It also explains why a coarse canvas, piecewise-constant as it is, already decodes to semantically meaningful content: the region mean itself carries that region's semantics and colour.

2. Hierarchy-conditioned one-step flow matching: push "the previous level" into the channel dimension so the level count stops multiplying the denoising steps

The most tangible cost of progressive generation is inference: multi-step denoising per level requires L×T network evaluations. TF instead performs exactly one evaluation per level, treating the average velocity field of the flow model as the carrier that jumps straight from noise to that level's target. Concretely, the backbone is a DiT shared across all levels; at training time a level index l is drawn uniformly for each batch element, and beyond the standard conditions c (class label y, time interval [r,t], and the guidance scale ω sampled at training time, since iMF folds CFG into training) the network receives three hierarchy-specific inputs: the noised current-level canvas \(z^{(l)}(t)=(1-t)z^{(l)}+t\epsilon\), the previous-level canvas \(z^{(l-1)}\), and the level index l. For l=0 the conditioning canvas is set to zero, so the coarsest level degenerates into generation driven by class conditioning alone — consistent with the intuition of "settling the big picture first, from nothing."

One engineering trade-off here is worth remembering: iMF's original recipe supplies the previous-level canvas by in-context conditioning, appending the conditioning tokens to the input sequence for attention to read, at the price of doubling sequence length. TF instead fuses in the channel dimension — both inputs are embedded to D dimensions, concatenated to 2D, and projected back to D by a linear fusion layer, leaving sequence length untouched. The level index goes through a learned embedding table and is injected as an extra conditioning token alongside the class and guidance embeddings. The prediction target follows pixel MeanFlow's x-prediction parameterization: the network directly predicts the denoised latent \(\hat{\mathbf{x}}_\theta\), the average velocity is recovered as \(u_\theta=(z^{(l)}(t)-\hat{\mathbf{x}}_\theta)/t\), and the compound prediction \(V_\theta=u_\theta+(t-r)\cdot\mathrm{JVP}_{\mathrm{sg}}\) is constructed in iMF style and regressed against the guided velocity. x-prediction is especially natural in DINOv2 space because the manifold hypothesis has a ready landing spot here: the denoised output lies on a structured representation manifold, which is far easier to regress than a noisy velocity field.

3. Structural loss: a hard constraint that every intermediate state must "look like the teacher"

Jumping from pure noise to a piecewise-constant target such as "two colour blocks for object and background" in a single step is not easy on flow loss alone — the model readily smears detail into that step and blurs the semantic distinction between levels. The structural loss targets exactly this: for every token it penalizes the deviation between the prediction and the target mean of its assigned region, using a squared cosine distance.

\[\mathcal{L}_{\text{struct}}=\frac{1}{K_l}\sum_{k=1}^{K_l}\frac{1}{|R^{(l)}_k|}\sum_{i\in R^{(l)}_k}\left(1-\cos\!\left\langle \hat{\mathbf{x}}_{\theta,i},\ \boldsymbol{\mu}^{(l)}_k\right\rangle\right)^{2}\]

(⚠️ Formula extraction in the cached text is misaligned; this is reconstructed from the paper's prose description, "squared cosine distance between the predicted token and its region's target mean." Refer to the original paper for coefficients.) It is applied only for l ∈ {0,…,L−2} and disabled at the finest level, which has no region structure to respect. \(K_l\) is the number of valid regions at that level and λ weights the term in the total objective, with λ=1 by default in the main text.

This term is the most literally "forcing" part of the method: it turns "what this level should look like" from an expectation into a differentiable supervisory signal, nailing down each level's semantic identity with the teacher hierarchy. It does two jobs at once — it guarantees that coarse canvases really are piecewise-constant semantic blocks (otherwise the intermediate state a user inspects degrades into a blur), and it encourages parent-child containment across levels already during training, rather than leaving structural drift to be discovered later by PMR at evaluation time.

4. Hierarchical sampling and trajectory-level editing: which level you edit decides how far the change travels

At inference the rules are simple: l=0 is generated from pure noise, \(\hat{\mathbf{z}}^{(0)}:=\hat{\mathbf{x}}_\theta(\epsilon, l{=}0, \mathbf{0}; c)\); every subsequent level draws fresh noise and is generated conditioned on the previous level's output, \(\hat{\mathbf{z}}^{(l)}:=\hat{\mathbf{x}}_\theta(\epsilon, l, \hat{\mathbf{z}}^{(l-1)}; c)\). One NFE per level, four in total, and each output can be handed to the RAE decoder immediately to become a viewable image.

Editing happens on that viewable image. The user modifies the intermediate result at some level l* to obtain \(\tilde{\mathbf{z}}^{(l\*)}\), after which only levels l>l* are regenerated while all coarser levels are kept as they were. The paper offers two complementary operations: feature editing replaces the mean feature of a selected region with a feature drawn from another region (or another image), relying on the fact that semantically similar content maps to nearby points in DINOv2 space to transfer the source region's semantic identity to the target; shape editing leaves features alone and only changes region boundaries, reassigning tokens along the border between adjacent regions to enlarge, shrink, or reshape a part's contour.

What is genuinely interesting is the built-in edit scope control. Because generation is Markovian (each level sees only the preceding one), an edit only propagates forward: the coarser the level you touch, the more downstream stages are affected and the more global the semantic consequences (swapping the object/background partition forces the composition of every later level to shift); the finer the level, the more local the effect (reshaping one subpart barely disturbs anything else). Users need no knowledge of the model internals — knowing simply "which level I acted on" lets them predict how far the change will reach. This is the most practical value TF has over endpoint-centric generation, and it directly motivated the dedicated metrics in the next design point.

5. Trajectory-aware metrics: turning "controllability" from a qualitative claim into two computable numbers

Metrics such as FID measure distributional quality and are blind to whether an edit only affects what it should affect, and to whether structure stays coherent across levels — precisely the two properties TF stands on. The paper therefore defines its own metrics.

The first is the Latent Invariance Score (LIS), which measures whether an edit disturbed regions it should not have: tokens at edited positions are marked 1 in a mask M, and only unedited positions are kept from the before/after regeneration to compute an average cosine distance.

\[\text{LIS}^{(\ell)}=\frac{1}{|\Omega_{\text{unedit}}|}\sum_{(i,j)\in\Omega_{\text{unedit}}}\left(1-\cos\!\left\langle \hat{\mathbf{z}}^{(\ell)}_{i,j},\ \hat{\mathbf{z}}'^{(\ell)}_{i,j}\right\rangle\right)\]

Lower scores mean unedited regions stay put more firmly. It is computed at every downstream level ℓ≥ℓ*, so one can see how an edit's influence diffuses level by level.

The second is the Parent Misrouting Rate (PMR), which measures whether structure drifts between adjacent levels: ideally a child-level region should fall entirely inside a single parent region, and one straddling several parents signals drift. The procedure first clusters the generated outputs at both levels using the pipeline of Sec. 4.1, assigns each child region a parent by spatial majority vote, and lets every token in that child region inherit the majority-voted parent as its expected parent. Two complementary quantities are then counted — spatial PMR, the fraction of tokens whose actual parent-level cluster assignment disagrees with the expected parent (i.e. whether child regions are cleanly contained inside single parents), and feature PMR, the fraction of tokens whose feature lies closer to an incorrect parent centre than to the assigned one (i.e. whether generation pushed a token's semantics away from its parent region's semantic mode). Both are lower-is-better and are computed for every consecutive level pair.

Loss & Training¶

The total objective is a weighted sum of the flow loss and the structural loss, \(\mathcal{L}=\mathcal{L}_{\text{flow}}+\lambda\mathcal{L}_{\text{struct}}\), with λ=1 by default in the main text (the λ ablation lives in the supplementary; the main text reports no numbers for it). The flow part follows the pixel MeanFlow recipe: x-prediction with a v-loss parameterization, the network outputs \(\hat{\mathbf{x}}_\theta(z^{(l)}(t), l, z^{(l-1)}; c)\), and the compound prediction \(V_\theta\) is regressed against the guided velocity \(v_g\), i.e. \(\mathcal{L}_{\text{flow}}=\mathbb{E}_{t,r,l}\lVert V_\theta-v_g\rVert^2\), with CFG injected during training (the iMF approach) so that the guidance strength can be tuned freely at inference without retraining.

For the implementation, the backbone is a DiT with patch size 16×16 (denoted TF/16) and all levels share one set of parameters; the optimizer is Muon at a constant learning rate of 1e−2; the ImageNet default is λ=1 with L=4 levels. Beyond the backbone the paper offers an optional post-training stage using an Inception-space Fréchet distance loss (FD-loss); every "+FD-loss" row in the main table comes from it (⚠️ the post-training details are in supplementary App. A.1; the main text only states that it is an Inception-space post-training).

Key Experimental Results¶

Main Results¶

Evaluation is on class-conditional ImageNet 256×256: each image is produced by L=4 one-step stages in DINOv2 feature space, intermediate outputs are rendered by a pretrained ViT-XL RAE decoder, and FID and IS are computed on 50k samples. The table below selects representative methods from the paper's Table 1.

Method	NFE	Space	Params	Epochs	FID↓	IS↑
JiT-L/16 (Heun)	100×2	pixel	459M	200	2.36	298.5
DeCo-XL/16	100×2	pixel	682M	320	1.90	303.0
LF-ViT/L (Heun)	50×2	pixel+DINOv2	465M	200	2.48	–
DiT-XL/2 (cfg=1.50)	250×2	SD-VAE	675M	1400	2.27	278.2
SiT-XL/2 (cfg=1.50)	250×2	SD-VAE	675M	1400	2.06	270.3
RAE+DiTDH-XL/2	50×2	DINOv2	839M	800	1.13	262.6
VAR-d20	250	VQ-VAE	600M	250	2.57	302.6
NVG-d20	184	VQ-VAE	497M	250	2.44	310.4
pMF-L/16	1	pixel	411M	320	2.52	262.6
iMF-M/2	1	SD-VAE	174M	640	2.27	–
iMF-L/2	1	SD-VAE	409M	640	1.86	–
TF-H/16 + FD-loss (ours)	4×1	DINOv2	1.1B	80	1.98	261.6

⚠️ In the cached table the parameter counts of TF-B/16 and TF-L/16 are both listed as 177M, and the "+FD-loss" rows as 515M, which looks like a column shift during text extraction (by scale, B/L/H are plausibly 177M/515M/1.1B); refer to the original paper for exact parameter counts. TF's FID/IS are computed with CFG, and "4×1" in the NFE column means 4 levels with one evaluation each.

Ablation Study¶

FD-loss post-training (same model, same training budget):

Config	NFE	FID↓	IS↑	Note
TF-B/16	4×1	7.93	221.4	80 epochs, no post-training
TF-B/16 + FD-loss	4×1	2.52	245.6	FID down 68%, IS up only 24.2
TF-L/16	4×1	6.38	248.1	80 epochs, no post-training
TF-L/16 + FD-loss	4×1	2.02	259.3	FID down 68%, IS up only 11.2
TF-H/16	4×1	5.97	256.8	80 epochs, no post-training
TF-H/16 + FD-loss	4×1	1.98	261.6	FID down 67%, IS up only 4.8

Behavior of the trajectory-aware metrics (Fig. 6; the original gives curves only, no numeric table):

Metric	Definition	Observation reported in the paper
LIS (edit invariance)	Mean cosine distance over unmasked tokens after editing; lower is more local	Clear level dependence: editing the object/background level gives the highest LIS at the final level, then parts, then subparts; axis range 0–0.20
Spatial PMR (child containment)	Fraction of tokens whose parent-level assignment disagrees with their child region's majority-voted parent	Computed per consecutive level pair; near zero for every pair → child regions are cleanly contained in single parents
Feature PMR (parent semantic mode)	Fraction of tokens whose feature is closer to an incorrect parent centre	Slightly higher than spatial PMR and rising with depth (finer semantic partition), but still low overall; axis range 0–0.12

⚠️ These two metrics are evaluated on TF alone: the paper states explicitly that no baseline produces semantic-region intermediates (NVG uses binary splits over discrete tokens with no region structure), so the numbers show only TF's internal self-consistency and cannot be compared across methods. The ablation of the structural loss weight λ is in the supplementary; the main text gives no numbers.

Key Findings¶

Under an identical 80-epoch budget, TF converges markedly faster early on. The same-epoch small-model references in the original Table 1 are DiT-B/2 (FID 43.47) and SiT-B/2+REPA (FID 24.40), against TF-B/16 at 7.93. This is the only comparison in the table where both training epochs and parameter scale line up, and it is the strongest support for the advantage of a coarse-to-fine trajectory in a representation space; conversely, comparing against RAE+DiTDH-XL/2 (1.13 after 800 epochs) on FID is neither fair nor favourable to TF.
FD-loss post-training is the single largest intervention, and its effect on FID and IS is strikingly asymmetric. On H/16, FID drops from 5.97 to 1.98 (about 67%) while IS moves only from 256.8 to 261.6. The post-training mainly improves distributional fidelity, whereas IS tracks class confidence, and the two do not always move together. The paper itself states that TF's primary contribution is not FID optimization but trajectory-level controllability — no other method in that table offers comparable intermediate states.
Scaling gains are modest without post-training and saturate quickly with it. Without FD-loss, B→L→H gives 7.93 → 6.38 → 5.97; with post-training, 2.52 → 2.02 → 1.98. Three model sizes buy roughly 0.5 FID, suggesting the bottleneck lies in the fixed-depth hierarchy and the single-step prediction rather than in parameter count.
The NFE advantage is structural. Among progressive/granularity-based generators, NVG-d20 needs 184 evaluations (9 content steps + 7×25 Euler steps) while TF-H/16 needs 4; NVG also relies on a custom multi-granularity VQ-VAE, whereas TF works directly on pretrained DINOv2 features and needs no task-specific tokenizer.
The trajectory-aware metrics give one positive and one cautionary result. LIS rises systematically as the edited level becomes coarser, confirming at the latent level that edit scope is genuinely controllable; spatial PMR near zero shows child regions essentially fall inside a single parent. Feature PMR, however, rises at deeper levels (parts→subparts, subparts→finest), meaning the parent's semantic constraint weakens as granularity gets finer — consistent with the Markovian design in which each level is conditioned only on the previous one, so coarse information can only travel down the chain and attenuates on the way.
Sharing one decoder is a justified choice, not a shortcut. The paper notes two reasons against per-level decoder fine-tuning: a shared decoder keeps decoded images from all levels in one consistent visual space, which is what lets a user compare the preview at level l* against the final image; and there is by construction no supervision for per-level tuning, since level canvases are piecewise-constant abstractions in feature space and no ground-truth intermediate images exist.

Highlights & Insights¶

A hard evaluation problem is converted into a design goal. Rather than fighting for FID, the paper first admits that no existing metric captures editing controllability and then defines LIS and PMR itself. The clever part is that neither needs extra annotation: LIS only needs an edit mask, and PMR reuses the clustering pipeline of Sec. 4.1. Anyone working on editable generation can copy this move — turn "does the change only affect what it should" into a computable cosine distance.
The choice of representation space solves two problems at once, and it is the fulcrum of the whole paper. DINOv2 features must both cluster well (unsupervised clustering recovers object-part-subpart) and decode back (the RAE decoder reconstructs images). The authors treat these two conditions as an entry bar for the substrate rather than as a patch applied afterwards — and this is the fundamental split between TF and pixel-space or VQ-VAE hierarchy methods.
Decoupling the number of levels from the number of denoising steps makes "multi-stage" affordable again. As long as each level needs multi-step denoising, L levels cost L×T, so richer hierarchies are more expensive; one step per level pushes the cost to L, so adding another semantic step is nearly free at inference. That rewriting of the trade-off transfers to any progressive generation framework.
Channel-dimension fusion instead of in-context conditioning is a cheap but necessary engineering decision. Concatenating the previous-level canvas with the noised current-level canvas in the channel dimension and projecting back preserves the previous level's information without doubling sequence length. For multi-stage conditioning where conditions and inputs share a shape, this is the first trick to try.
The Markovian chain is both the source of the method's strength and of its weakness, and the authors say so. It buys predictable edit scope (propagation only forward) at the cost of coarse information reaching fine levels only through an attenuating chain. Writing out both sides of a design like this is worth emulating.

Limitations & Future Work¶

The hierarchy itself is the weakest link. It comes from fixed-depth unsupervised clustering, and the object/background split depends on a centre prior ("the cluster closer to the image centre is the object"). For images whose subject is off-centre, or that have no single dominant object (landscapes, textures, crowds), the first level can split wrongly, and a wrong coarse level then affects every downstream level through the chain. The paper's suggested fix is a richer hierarchy source such as concept segmentation models (SAM 3) or an adaptive hierarchy depth per image.
The evaluation surface is narrow and the conclusions should be extrapolated with care. All experiments are class-conditional ImageNet 256×256, with no text conditioning, no other datasets, no resolution variation; controllability rests on latent-space metrics (LIS/PMR) plus decoded-space checks in the supplementary, with no user study verifying that people can actually steer the trajectory as intended. Moreover, LIS/PMR are measured on TF alone with no baseline, so "edit scope is controllable by level" is currently self-consistency evidence rather than comparative evidence.
The paper itself calls the results early-stage. The main text reports 40/80 epochs with longer training and scaling analysis deferred to the supplementary, and the best FID, 1.98, is clearly behind RAE+DiTDH-XL/2 (1.13) trained in the same DINOv2 space — at the cost of 800 epochs and 50×2 NFE.
Single-step prediction carries a lot of weight, and coarse levels can "detail too early." Each level jumps from pure noise in one step, so the model must land exactly at that level's semantic granularity. The structural loss mitigates this, but it only constrains each token's deviation from its region mean, not the relations between regions (e.g. whether the relative placement of parts is sensible). An additional inter-region relational constraint at coarse levels is a natural extension.
Improvement directions: extend edit scope from "level" to a combination of level and spatial mask; use intermediate levels for cheap search (pick the best of several l=2 candidates before generating downwards, far cheaper than best-of-n in pixel space); transfer the "structural teacher + one step per level" recipe to video (coarse-to-fine in time: motion outline first, texture later) or 3D.

vs NVG (Next Visual Granularity): the closest in goal — both decompose an image into explicit granularity levels and generate them progressively. They differ on three axes: substrate, where NVG depends on a custom multi-granularity VQ-VAE while TF uses pretrained DINOv2; hierarchy construction, where NVG uses greedy binary splits (k=2) in VQ-VAE latent space with a depth pinned by resolution (1+log2(16×16)=9) and intermediates lacking clear semantic grouping, while TF clusters in feature space with a fixed depth of 4 and clearly semantic levels; and inference, where NVG needs 184 NFE and TF needs 4. The trade-off is that NVG's levels are denser, while TF's levels are semantically strong but coarse.
vs Latent Forcing / Diffusion Forcing / DeCo: all three exploit trajectory structure to improve the final image — Latent Forcing reveals latent features before pixel detail (and discards them), Diffusion Forcing varies per-token noise levels to change generation order, DeCo factorizes pixel diffusion by frequency. Their intermediate states are implicit or one-shot computational scaffolding; TF keeps the intermediate state, gives it a decoder, an editing interface, and metrics. TF's contribution is thus not "trajectory structure helps" but "trajectory structure can serve as an interface."
vs RAE / REPA: RAE supplies the two things TF depends on — a representation geometry with well-organized semantics and a decoder that reconstructs images from that space; the representation alignment hypothesis behind REPA explains why plain clustering on DINOv2 recovers object-part-subpart. TF stands on their shoulders: RAE generates in a single forward pass, while TF splits generation in the same space into semantically ordered steps and keeps them editable.
vs layout-conditioned generation (GLIGEN / ControlNet style): ⚠️ these lines are not cited by the paper; the following is my own mechanism-level distinction. Layout-conditioned generation treats layout as an external input — the user draws boxes, masks, edge maps or depth maps, and a pretrained multi-step model fills in appearance; once given, the condition never changes, and the intermediate states remain invisible and uneditable. In TF the layout is not an input but the coarsest levels of the trajectory itself: the model generates the object/background partition from noise, the user can inspect it, edit it, and continue; control happens during generation rather than before it, and the editing unit is a semantic object (a region's feature or boundary) rather than a redrawn condition map. The two express different interface philosophies: "specification → image" versus "trajectory state → edit → continue generating."
vs VAR / FlowAR: they organize progressive generation along spatial resolution (coarse resolution first, finer later), whereas TF's levels are organized by semantic granularity at a fixed spatial resolution (object → part → subpart). The two axes are orthogonal and could in principle be stacked: semantics first, resolution second, or multi-scale refinement within each level.

Rating¶

Novelty: ⭐⭐⭐⭐ The pivot to treating the generative trajectory as a user interface is clear, and combining a clustered hierarchy with one-step per-level conditioning is genuinely new; but hierarchical generation itself is not new, and the difference from NVG lies mostly in substrate and cost rather than in paradigm.
Experimental Thoroughness: ⭐⭐⭐ The main table spans three families of baselines with clearly stated protocols, and the FD-loss control plus the trajectory-aware analysis are solid; however, there is a single setting (ImageNet 256×256), only 40/80 epochs are reported, the λ and scaling ablations sit in the supplementary, and the controllability metrics have no baseline to compare against — not enough empirical weight to fully carry the central claim of trajectory-level control.
Writing Quality: ⭐⭐⭐⭐ The motivation chain (coarse-to-fine human painting → why intermediates must be semantic and decodable → substrate choice) is told clearly, and the paper volunteers the cost of the Markovian conditioning and its rationale for a shared decoder; points come off for the badly misaligned formula extraction and the questionable parameter column.
Value: ⭐⭐⭐⭐ Directly useful for anyone working on controllable generation, interactive creation, and progressive generation, and the LIS/PMR way of quantifying controllability transfers on its own; the current FID and the narrow setting make it more the opening of a direction than a system ready to deploy.