WorldCache: Content-Aware Caching for Accelerated Video World Models¶

Conference: ECCV 2026
Paper: ECCV page
Code: https://github.com/umair1221/WorldCache
Area: Video Generation / Inference Acceleration
Keywords: feature caching, diffusion transformer, motion compensation, saliency weighting, world models

TL;DR¶

Without retraining the video diffusion model, WorldCache improves both cache-hit decisions and feature approximation, reducing latency from 55.04 to 24.48 seconds in a Cosmos-2B image-conditioned generation experiment while retaining a PAI-Bench Overall score of 0.7977 versus 0.8030; the paper reports 2.3× acceleration, although its speedup and quality-retention figures require attention to rounding conventions.

Background & Motivation¶

Video world models rely on the spatiotemporal attention of diffusion transformers (DiTs) to generate future visual states, but executing a deep network at every denoising step makes inference expensive. Intermediate activations at neighboring steps often contain redundancy. Training-free feature caching therefore uses a small amount of shallow computation to estimate change and skips deeper computation when that change is sufficiently small. FasterCache employs a fixed reuse schedule, whereas DiCache uses shallow probes and historical residual alignment to make caching an online decision.

World models need to preserve motion and interactions, which cannot be assessed solely through average changes over an entire feature map. Small moving objects can be overwhelmed by static backgrounds; even when reuse is permitted, misaligned historical features can create ghosting or smeared boundaries. High-noise layout formation and low-noise texture refinement also differ in their sensitivity to approximation error, making a single fixed threshold difficult to tune for both fidelity and speed. The challenge is not merely to skip more computation, but to coordinate where, when, and how approximation is used.

The authors organize these issues as “perception-constrained dynamical caching.” However, DiCache already mixes historical residuals, and TaylorSeer forecasts future features, so prior methods cannot all be characterized as copying unchanged snapshots. Core idea: use input changes and spatial saliency to decide when recomputation is worthwhile, improve reuse through direction-aware residual blending and optional spatial alignment, and obtain additional cache hits by relaxing thresholds late in denoising.

Method¶

Overall Architecture¶

Each denoising step first executes several initial DiT blocks as a probe, then either runs the remaining deep blocks or approximates their output from cached history. CFC supplies a motion-related base threshold, SWD supplies a spatially weighted drift signal, and ATS adjusts the threshold according to denoising progress. OFA produces an approximate output only on a cache hit. A cache miss executes the deep blocks and refreshes the residual cache; both paths return to the sampler for the next denoising step.

Step indices in the paper refer to diffusion denoising progress, not video frame numbers. Each input latent contains multiple video frames, so changes between denoising steps should not be equated directly with object motion between adjacent video frames. Both CFC and optional warping derive signals from latent changes across denoising iterations.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    I["Current latent<br/>and historical cache"] --> P["Execute shallow probe"]
    I --> C["CFC Motion-Adaptive Threshold"]
    P --> S["SWD Saliency-Weighted Drift"]
    C --> A["ATS Denoising-Stage Scheduling"]
    S --> G{"Weighted drift below<br/>the scheduled threshold?"}
    A --> G
    G -->|Yes: cache hit| O["OFA Aligned Residual Approximation"]
    G -->|No: cache miss| F["Execute deep blocks<br/>and refresh residual cache"]
    O --> Y["Deep output or its approximation<br/>returned to the sampler"]
    F --> Y

CFC and SWD provide parallel decision signals, ATS adjusts only the threshold, and OFA operates only on the hit branch. There is no training-supervision branch because the method neither updates model weights nor requires additional training data. Its components still incur runtime overhead.

Key Designs¶

1. CFC Motion-Adaptive Threshold: reduce risky reuse when the input changes substantially

Causal Feature Caching (CFC) computes the normalized change between raw latent inputs separated by two denoising steps, using it as the authors’ “velocity” proxy. Larger changes tighten the caching threshold, making it harder for shallow drift to satisfy the skip condition; smaller changes bring the threshold toward its preset base value. This does not estimate true physical velocity. Instead, it provides a low-cost indicator of rapid latent evolution to avoid reusing stale deep states.

The authors motivate a two-step rather than one-step gap by noting that the immediately preceding step may already contain an approximation. Two cache slots alternate according to step parity. One implementation issue remains unresolved in the supplied text: the paper describes the two-step-old state as the latest fully computed anchor, but does not fully explain how that property always holds when consecutive cache hits are allowed. Two slots alone do not establish that two recent, fully computed historical steps are available at every point.

Cached Equations (2) and (3) are damaged, particularly the operators defining the threshold function. This note therefore does not guess whether it uses exponential decay or another form. The surrounding prose supports normalized two-step change, a tighter threshold for higher velocity, and separate base-threshold and motion-sensitivity parameters.

2. SWD Saliency-Weighted Drift: give structurally complex regions more influence over the decision

Saliency-Weighted Drift (SWD) constructs spatial weights from features already produced by the probe, without an additional saliency network. It first averages probe outputs over the batch and video-time axes, computes channel variance at each spatial position, and normalizes the resulting saliency map to the interval from 0 to 1. High channel variance often corresponds to edges, textures, and structurally complex regions. Figure 3 provides visualization support, but this is not semantically supervised foreground segmentation and does not guarantee that every important interacting object is highlighted.

When measuring changes between shallow features at neighboring steps, SWD amplifies contributions from information-rich positions and aggregates them spatially. The paper explicitly gives the weight as \(1+\beta_s\hat S\). With nonnegative saliency weights, this does not reduce absolute background weights below 1. “Background attenuation” should therefore mean reduced importance relative to highly salient regions, not an explicit reduction of every background error term. The authors argue that this avoids background-dominated decisions; the ablation supports faster execution after adding SWD, but does not prove that every foreground change is detected correctly.

Temporal averaging introduces another boundary: the final spatial saliency map combines multiple latent frames and can smooth out small, rapidly moving targets. SWD is thus an inexpensive perceptual proxy rather than precise object tracking. It changes which regions dominate the decision, not the model’s underlying video-motion representation.

3. ATS Denoising-Stage Scheduling: permit more cache hits toward the end of denoising

Adaptive Threshold Scheduling (ATS) applies a relaxation factor that grows with denoising progress to the threshold supplied by CFC. Early steps primarily establish layout and motion, so the threshold remains relatively tight. Later steps mainly refine texture, and a larger threshold allows some changes that would otherwise trigger recomputation to be handled by approximation. ATS does not reduce the sampler’s total number of steps; it lets more steps execute only the probe and approximation components.

The main text gives an example with 35 steps and scheduling parameter 4.0: the multiplier is approximately 1.2 at step 2 and approximately 4.6 at step 32. Figure 4 uses a fixed threshold of 0.12 and reports skip rates of 36% before scheduling and 68% with scheduling. These are not universal skip rates for all tasks under the default base threshold of 0.08. Equation (13) also has missing symbols in the extraction, so this note does not present an expression inferred from the examples as the authors’ exact formula.

ATS can be understood as using the quality margin supplied by the preceding controls and the subsequent OFA approximation. It does not prove that the low-noise stage is necessarily safe; it identifies a faster operating point on the tested models and tasks. Its quality impact must be measured on final generations rather than guaranteed solely by denoising progress.

4. OFA Aligned Residual Approximation: account for historical change direction and spatial misalignment after a cache hit

The temporal part of Optimal Feature Approximation (OFA) is called Optimal State Interpolation (OSI). It caches the computational residual between deep output and raw input, rather than simply caching a final video. When a hit prevents full deep execution, it obtains a partial residual from the shallow probe and uses its difference from a historical residual as a target. It then estimates how far to advance along the recent historical residual-change direction. Unlike DiCache’s scalar coefficient derived from L1 distance ratios, this uses least-squares vector projection, retaining directional information through an inner product.

When current changes no longer follow the historical direction, projection can reduce the gain applied along that old direction. The authors clip this gain to the interval from 0 to 2 to limit amplification from small denominators. A combination of historical residuals is then added to the current input to approximate the deep output. Here, “optimal” applies only to the chosen one-dimensional residual direction and shallow proxy target—not to the full feature space, true deep output, or perceptual quality. Damaged extraction of Equations (8)–(10) prevents complete verification of the exact notation.

The spatial part is optional motion compensation. Multiscale correlations between the current and preceding denoising-step latents estimate a displacement field without an external optical-flow network. Cached deep features are transformed toward the current coordinates before residual combination. The paper reports less than 3% overhead per cached step for this component and disables it during the first 5 denoising steps because displacement estimation is unreliable under high noise. This local overhead must not be equated with the end-to-end impact of the complete OFA module: adding OFA increases total latency in the ablation.

Loss & Training¶

There is no new training objective. The tested Cosmos-Predict2.5 configuration uses 35 Euler steps and generates 93 frames at 16 FPS, approximately 5.8 seconds, corresponding to 24 latent frames. The 2B and 14B checkpoints remain unchanged. WAN2.1 uses the official inference configuration for each checkpoint; Cosmos frame counts and step counts should not be automatically attributed to the other models.

The default base threshold is 0.08, motion sensitivity 0.2, saliency weight 0.12, and stage-scheduling parameter 4.0. The main text states that these parameters are fixed across models and tasks, without per-prompt tuning. End-to-end latency is measured on one NVIDIA H200 with matching batch and precision settings across methods. The supplied main text does not fully specify probe depth, cache initialization, or every setting controlling optional warping.

Key Experimental Results¶

Main Results¶

PAI-Bench reports Domain scores for physical evaluation, Quality scores for visual quality, and their mean as Overall. The following reproduces the Cosmos-Predict2.5-2B / Image2World results from the paper’s Table 4, without mixing in the slightly different baselines from Tables 2 or 6. Speedups are the paper’s reported values, not recalculated values.

Method	Domain ↑	Quality ↑	Overall ↑	Latency (s) ↓	Reported speedup ↑
Baseline	0.8450	0.7610	0.8030	55.04	1.0×
EasyCache	0.8399	0.7552	0.7975	40.25	1.37×
DiCache	0.8352	0.7522	0.7941	39.68	1.39×
TeaCache (Fast)	0.8381	0.7549	0.7965	41.00	1.34×
TeaCache (Slow)	0.8396	0.7562	0.7979	49.59	1.10×
MagCache	0.7682	0.7410	0.7546	26.87	2.05×
TaylorSeer	0.7106	0.6956	0.7031	29.49	1.86×
WorldCache	0.8395	0.7559	0.7977	24.48	2.30×

WorldCache has the lowest latency in this table, but not the highest Overall score among accelerated methods: TeaCache (Slow) scores 0.7979, higher by 0.0002. The important result is substantially lower latency at nearly the same quality level.

From the displayed values, 55.04 / 24.48 is approximately 2.25 rather than exactly 2.30, while 0.7977 / 0.8030 corresponds to approximately 99.34% Overall retention. The abstract reports “2.3× and 99.4%.” These should therefore be treated as the paper’s summary figures, not exact retention values established from the rounded table entries.

Cross-model evidence includes Table 3: WAN2.1-1.3B T2W decreases from 120.04 to 50.84 seconds, with Overall changing from 0.7727 to 0.7721; WAN2.1-14B I2W decreases from 475.60 to 206.73 seconds, with Overall changing from 0.7384 to 0.7388. The two settings report 2.36× and 2.30×, respectively. They are not a model-size ablation under an identical task.

Ablation Study¶

Table 6 incrementally adds modules on Cosmos-2B / I2W. Its baseline Overall is 0.8027, rather than Table 4’s 0.8030; the values below preserve the original table’s figures.

Config	Domain ↑	Quality ↑	Overall ↑	Reported speedup ↑	Latency (s) ↓
Base	0.8447	0.7607	0.8027	1.00×	55.04
+ CFC	0.8457	0.7583	0.8020	1.52×	36.06
+ CFC + SWD	0.8414	0.7592	0.8003	1.70×	33.24
+ CFC + SWD + OFA	0.8468	0.7602	0.8035	1.47×	37.38
+ CFC + SWD + OFA + ATS	0.8395	0.7559	0.7977	2.3×	24.48

This is an incremental ablation, not a controlled removal of each component from the full model. It supports the observation that adding OFA improves quality while reducing speed, followed by acceleration from ATS. It does not establish that removing any individual module necessarily breaks the complete method.

For the SWD row, 55.04 / 33.24 is approximately 1.66, differing from the reported 1.70. The full model’s Overall reduction from this table’s baseline is approximately 0.62%, whereas the main text calls it “within 0.6%.” These discrepancies should be preserved rather than silently editing the original numbers into agreement.

Key Findings¶

OFA is not directly an acceleration module. Adding it changes Overall from 0.8003 to 0.8035 but increases latency from 33.24 to 37.38 seconds. The authors attribute this to better approximation and a changed cache-hit pattern, creating a quality margin for subsequent scheduling.
ATS primarily uses approximation more aggressively. On top of the OFA configuration, latency falls from 37.38 to 24.48 seconds while Overall falls from 0.8035 to 0.7977. This is a speed–quality trade-off, not proof of lossless acceleration.
Standard video evaluation provides additional support. In Table 5, HunyuanVideo’s VBench score changes from 82.34 to 82.23 while latency decreases from 984 to 334 seconds, reported as 2.95×. Cosmos-2B changes from 76.35 to 76.47 and from 54.87 to 24.42 seconds, reported as 2.25×.
Small score gains do not establish better physical understanding. Some settings slightly exceed the uncached baseline, but the main text provides insufficient variance or significance analysis. These results indicate no obvious degradation at those evaluation points, not that caching teaches more accurate physical laws.

Highlights & Insights¶

Optimize both the decision and the approximation. Changing only the skip threshold can shift error into reused outputs; improving approximation alone can increase latency. The incremental experiment clearly shows why both should be evaluated together with stage scheduling.
Extract control signals from existing computation. Latent-input changes and probe channel variance come from the model itself, avoiding large additional motion or saliency networks. Their low cost also means they are proxies whose failure cases need examination.
Direction carries more information than magnitude alone. Residual projection considers whether current changes align with historical direction, supplying richer evidence than a distance ratio. Its optimality remains conditional on the proxy target and constrained direction.

Limitations & Future Work¶

Short-video metrics do not validate a closed-loop world model. The main Cosmos configuration generates approximately 5.8 seconds of video. The supplied text does not report sufficient long-horizon autoregressive rollout or real closed-loop decision experiments to quantify caching’s impact on long-term planning.
Saliency and motion proxies can fail. Channel variance highlights complex texture rather than necessarily important semantic objects, and temporal averaging can dilute small targets. Denoising-latent changes include both noise removal and content evolution, not pure motion.
Equations and cache timing require further verification. Several formulas are damaged in the supplied extraction. The relationship between two-step anchors and consecutive hits, along with warping settings, is not fully specified. This note does not replace source or typeset-paper verification with guessed equations.
Benefits depend on the inference configuration. The evidence covers the tested models and tasks on H200. Other hardware, probe depths, resolutions, and samplers can change the bottleneck. The main text claims fixed hyperparameters transfer, but does not provide a complete sensitivity analysis.
Further work could target measurable error constraints. Under a fixed latency budget, scheduling thresholds from cache-error feedback and separately testing spatial compensation versus temporal interpolation would better establish when aggressive reuse is justified.

Compared with DiCache: WorldCache inherits the shallow-probe/deep-skip framework and adds motion- and saliency-aware decisions, direction-sensitive residual projection, and denoising-progress scheduling. DiCache already combines historical states, so WorldCache is not the first caching method to avoid direct copying.
Compared with FasterCache: FasterCache exploits fixed temporal schedules and CFG-branch redundancy, while WorldCache emphasizes content-dependent decisions. In Table 1’s Cosmos-2B / T2W setting, FasterCache reports 34.5 seconds and Overall 0.67, versus WorldCache’s 26.3 seconds and 0.75—a better operating point in that setting.
Compared with TeaCache / EasyCache / MagCache / TaylorSeer: These improve caching through timestep signals, runtime changes, residual magnitudes, and feature forecasting, respectively. WorldCache combines motion proxies, spatial weighting, and alignment rather than establishing that all prior methods lack dynamic prediction.
Compared with video feature propagation: Latent warping continues the idea of aligning old features through displacement. Here the correspondence is between denoising iterations, requiring validation under high noise and large displacements.

Rating¶

Novelty: 4/5. Jointly redesigning cache decisions and approximation for dynamic video content has a clear task motivation; the individual tools and probe framework build on prior work.
Experimental Thoroughness: 4/5. Multiple model families, PAI-Bench and VBench, and several caching baselines provide breadth; incremental ablations and long-horizon closed-loop validation still have limitations.
Writing Quality: 3/5. The four components are logically organized, but speedup, retention, and cross-table baselines differ in presentation, while cache timing and some implementation details require verification.
Value: 4/5. Lower video-generation latency without retraining has clear engineering value; quality retention is an empirical result under specific evaluation settings, not a universal guarantee.