NeurIPS2026; Oral is supplied task metadata, and official acceptance status has not been independently verified Physics & Scientific Computing chaotic dynamics extreme events ensemble conditioning finite-time instability coarse-resolution correction

Mechanism-Aware Ensemble Conditioning for Data-Limited Emulation of Extreme Events¶

Conference: NeurIPS2026; Oral is supplied task metadata, and official acceptance status has not been independently verified
arXiv: 2609.30746v1
Area: Physics / Scientific Computing (physics)
Keywords: chaotic dynamics, extreme events, ensemble conditioning, finite-time instability, coarse-resolution correction

TL;DR¶

The paper compresses a reference-nudged stochastic coarse ensemble into dynamical-sensitivity context and uses FiLM to modulate existing temporal correctors, improving QG extreme-event area and frequency statistics with limited high-resolution training data without guaranteeing better global distributions or every tail diagnostic than a long-data baseline.

Background & Motivation¶

Extreme events in chaotic systems are not simply occasional large values drawn from a static distribution: trajectories temporarily enter sensitive regions, where perturbations amplify along finite-time growing directions and produce unusually large responses. A surrogate exposed to a short trajectory may learn common states without recognizing these transient amplification mechanisms. For climate and fluid simulations, long-horizon event frequencies, distribution tails, and exceedance areas can matter more than pointwise trajectory errors after prolonged evolution, because chaos destroys phase correspondence.

The available resources here are a cheap but biased coarse model and a small amount of expensive high-resolution reference data. Existing non-intrusive correction methods first use nudging to synchronize the coarse trajectory with the projected reference, avoiding direct pairing of exponentially separated trajectories, and then train a corrector such as STORN. Nudging primarily resolves training alignment, however, without informing the model how nearby perturbations grow. Simply stacking additional ensemble members also increases input dimensionality, making short-data training harder.

The authors instead treat the stochastic ensemble as a dynamical sensor, not as model-parameter uncertainty or an EnKF posterior. Under local small-noise conditions, covariance is governed by finite-time deformation kernels and can signal sensitive regimes; a low-dimensional summary then conditions the corrector's hidden features. Core Idea: use a reference-nudged ensemble during training to learn how sensitive regimes affect correction, then drive the same FiLM interface at test time with a causal proxy computed only from coarse-trajectory history, without modifying the coarse solver.

Method¶

Overall Architecture¶

The input is a coarse-model state sequence rather than a high-resolution solver run; the output is a corrected trajectory on the same coarse grid for long-horizon statistical analysis. Training additionally uses a projected reference and an auxiliary stochastic ensemble: the reference-nudged ensemble supplies context summaries, while the projected reference supplies supervision. Deployment does not access the reference or pretend that the instantaneous training-ensemble covariance is directly available as a test input.

The two instantiations share “reference-nudged ensemble → instability summary → FiLM correction,” but use different summaries and backbones. The low-dimensional example explicitly uses covariance features with a gated residual-attention model; QG retains only layerwise amplitude statistics with a stochastic temporal network, STORN. The causal deployment proxy replaces the source of context, not the coarse model, and is not an additional high-resolution predictor.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    R["Training: projected reference<br/>and coarse model"] --> A["Reference-nudged ensemble"]
    A --> B["Instability summary"]
    T["Test: free-running<br/>coarse-trajectory history"] --> C["Causal deployment proxy"]
    B -->|Training context| D["FiLM correction"]
    C -->|Test context| D
    R -.->|Training supervision only| O["Corrected trajectory"]
    D --> O

Key Designs¶

1. Reference-nudged ensemble: evolve perturbations around a comparable base trajectory

The high-resolution state is first projected into coarse space. The coarse model relaxes toward this reference, with a shorter relaxation timescale imposing stronger nudging; stochastic members add independent Brownian noise to the same nudged dynamics. Members do not interact directly, and sharing a reference does not mean sharing noise. Written in deviation coordinates, the paper's Eq. (1) is:

\[ \mathrm{d}Q^{(m)}(t)=\left[f\bigl(u(t)+Q^{(m)}(t)\bigr)-\mathcal{P}F\bigl(\mathbf{u}(t)\bigr)-\tau^{-1}Q^{(m)}(t)\right]\mathrm{d}t+\sqrt{2\beta^{-1}}\,\mathrm{d}B_t^{(m)}. \]

Here \(u=\mathcal{P}\mathbf{u}\) is the projected reference, \(f\) is the coarse vector field, \(\tau\) is the nudging timescale, and \(\beta\) is the inverse noise level. Experiments instead use \(\sigma\) for perturbation amplitude; this notation differs from the QG beta-plane parameter and the FiLM bias. Synchronization reduces trajectory drift so that ensemble deformation more closely reflects how the same local dynamics amplify perturbations, rather than mixing unrelated trajectories.

In QG, noise enters only resolved Fourier modes, and members are integrated on the coarse grid. The reference uses a 128×128 grid and the coarse model a 24×24 grid, with identical governing equations; unresolved scales cause the main bias. Seven Gaussian bottom-topography bumps break spatial homogeneity and generate localized extremes; the nudging timescale is 16.0. The low-dimensional example instead adds linear damping to the third coordinate of the original three-dimensional system, deliberately weakening its strongest growth episodes, with a nudging timescale of 8.

2. Instability summary: explain the ensemble through deformation kernels, then compress it for the task

The theory first explains why this is not ordinary stochastic augmentation. Theorem 2.1 / Appendix A.2 requires a twice continuously differentiable coarse vector field that is globally Lipschitz with linear growth, bounded reference paths and projected reference derivatives on a fixed finite interval, and a globally bounded Hessian. It additionally assumes centered second and fourth moments with small-noise scales \(O(\beta^{-1})\) and \(O(\beta^{-2})\), respectively. The general moment boundedness in Appendix A.1 does not automatically establish these two scales.

Under these conditions, ensemble-mean drift differs from deterministic nudged drift by \(O(\beta^{-1})\). Covariance satisfies a perturbed Lyapunov equation, and its integral representation retains both propagated initial covariance and the deformation history generated by continuous noise injection:

\[ \begin{aligned} \dot\Sigma(t)&=L_\tau(t)\Sigma(t)+\Sigma(t)L_\tau(t)^\top+\frac{2}{\beta}I+R_\Sigma(t),\qquad \|R_\Sigma(t)\|_{\mathrm{op}}\le C_T\beta^{-3/2},\\ \Sigma(t)&=\Phi(t,0)\Sigma(0)\Phi(t,0)^\top+\frac{2}{\beta}\int_0^t\Phi(t,s)\Phi(t,s)^\top\,\mathrm{d}s+O_T(\beta^{-3/2}). \end{aligned} \]

Here \(L_\tau(t)=Df(u(t)+\bar Q(t))-\tau^{-1}I\), and \(\Phi(t,s)\) is its fundamental solution. The proof Taylor-expands around the mean and uses centering to eliminate the linear mean contribution; Itô's product rule yields the noise source, second and fourth moments control the cubic remainder, and variation of constants gives the integral expression. The conclusion is that covariance aggregates the same deformation kernels under small noise and finite time, not that it unconditionally equals the true unstable subspace. Alignment with a particular leading Cauchy–Green subspace additionally requires spectral-gap and scale-separation assumptions; an integral over kernels with different starting times cannot be freely replaced by a single instantaneous direction.

Finite-ensemble error control is a separate result. Theorem 2.2 / Appendix A.3 fixes one time and assumes iid members centered around the population mean, zero mean, a fourth moment bounded by \(\kappa\), and a population-covariance eigengap \(\delta>0\) for the target subspace. The leading-subspace projector error then satisfies:

\[ \mathbb{E}\|\widehat R(t)-R(t)\|_{\mathrm{op}}\le\frac{2\sqrt\kappa}{\delta\sqrt M},\qquad \mathbb{P}\!\left(\|\widehat R(t)-R(t)\|_{\mathrm{op}}\ge\varepsilon\right)\le\frac{4\kappa}{\delta^2M\varepsilon^2}. \]

The proof uses the Frobenius variance of independent random matrices, followed by Davis–Kahan and Markov inequalities. This is a fixed-time \(M^{-1/2}\) mean-error result, not a simultaneous high-probability guarantee over an entire interval or a claim that any tiny ensemble resolves a high-dimensional space. Experiments center around a sample mean, whereas the theorem uses population centering; their bounds should not be treated as literally the same rigorous result.

The low-dimensional model provides two explicit summaries: an eight-dimensional context containing three coordinate means, three standard deviations, and two leading eigenvalues, or leading eigenvectors with signs aligned over time. Comparing the leading covariance direction with optimally time-dependent (OTD) modes primarily checks co-activation during instability bursts, rather than exact componentwise amplitude agreement.

QG context has only four dimensions: each member first supplies the spatial mean of the absolute streamfunction amplitude in each layer, followed by an across-member mean and standard deviation, giving two numbers per layer. These are not eigenvectors of a spatial covariance matrix and do not establish recovery of a spatial unstable subspace. They are amplitude/spread proxies motivated by the theory. The spatial-average formula in Appendix D.2 uses a one-dimensional integral and normalization inconsistent with the two-dimensional grid description; this note retains the verbal definition instead of repairing it into a purported exact author equation.

3. Causal deployment proxy: avoid test-reference leakage and acknowledge context shift

Training context comes from a reference-nudged ensemble; test context comes from a historical rolling window of the free-running coarse trajectory. The low-dimensional example computes rolling statistics and eigendirections from coarse history, while QG computes layerwise amplitude statistics from coarse-field history. “Causal” here means that context uses only current and past coarse data, not that the paper performs causal inference or proves that the entire attention backbone uses a causal mask.

This substitution prevents high-resolution reference leakage but creates a genuine proxy distribution shift: across-member transverse spread becomes temporal coarse variability, and these quantities are not equivalent. Appendix D.5 explicitly rejects interpreting the test rolling statistic as an instantaneous transverse covariance estimate. Its usefulness depends on the coarse dynamics retaining identifiable signatures of localized sensitivity. The theoretical proof concerns a reference-nudged stochastic ensemble, not equivalence of the deployment proxy or direct guarantees of long-horizon tail generalization.

This distinction also clarifies non-intrusive deployment: the corrector processes coarse-solver outputs, and the reference supplies supervision and context construction during training; at deployment, the reference appears only in offline diagnostics. A training-reference edge must not enter test context, nor should corrected outputs be fed back into the coarse integrator while claiming that the paper changes the solver.

4. FiLM correction: condition hidden features instead of stacking all members

A small network maps the summary into hidden-channel scales and offsets, then applies affine modulation:

\[ (\gamma_t,\beta_t)=g_\theta(c_t),\qquad \widetilde h_t=(1+\gamma_t)\odot h_t+\beta_t. \]

This interface lets different sensitivity regimes trigger different corrections without using every ensemble member as a high-dimensional input. The low-dimensional model encodes coarse states and positions before a Transformer; FiLM follows the encoder and precedes gated residual output heads. Hidden width is 96, with 4 attention heads and 3 layers; the context-network width is 48. A gate-bias initialization of −2.0 keeps the initial prediction close to the coarse state instead of immediately imposing a large correction.

For QG, STORN encodes field states, parameterizes Gaussian latent variables and samples them by reparameterization, then passes encoded and latent features to an LSTM. FiLM modulates recurrent hidden states before a linear decoder produces the corrected field. Encoder, latent, and recurrent widths are all 60; context-embedding width is 32. Context dimensionality does not grow with ensemble size, but training-ensemble generation costs still do. Controlled comparisons retain the backbone, optimizer, and loss while adding the context interface and its parameters, making the information-source effect easier to isolate than replacing the entire network with a larger one.

A Worked Example¶

Consider the short-data QG setting: take 50 time units of high-resolution reference data, project them to a 24×24 grid, and generate a reference-nudged coarse trajectory with an auxiliary ensemble using noise amplitude 0.01. The main result labels the ensemble setting M=2, while appendices and figure captions use N_add=2; this note does not decide whether a base member is counted separately.

During training, ensemble amplitude means and standard deviations in each layer form context, STORN receives coarse windows, FiLM modulates hidden features, and the projected reference supplies targets. The coarse model is then run independently; at test time, only its history supplies rolling amplitude context for the trained corrector's long-horizon outputs. The test reference is used to calculate errors, not as an input to the context network.

Evaluation covers 50,000 time units. At each threshold, the fraction of exceeding grid cells is computed, followed by comparisons of its distribution and the frequency of any exceedance. These diagnostics assess event extent and occurrence even after chaotic trajectories lose pointwise phase correspondence.

Loss & Training¶

QG retains STORN's squared field-reconstruction error, an absolute spatial-sum penalty for each layer, and Gaussian-posterior KL regularization toward a standard-normal prior. The mass penalty is related to the domain-integrated layer-height perturbation represented by streamfunctions; it does not directly penalize tail frequency. FiLM and unconditioned STORN share the objective rather than introducing the evaluation exceedance-area distance as a new training loss.

QG uses Adam, batch size 32, and 2000 training epochs; windows default to length 100 and shrink when data are insufficient. The ML timestep is 0.1, with a temporal 90% / 10% train/validation split. The main text states that backbone and training settings are inherited from the long-data baseline, while the short-data FiLM ensemble parameters are fixed rather than selected separately for each evaluation diagnostic.

The low-dimensional shared objective combines tail-weighted path error, correction-amplitude regularization, window mean/standard-deviation matching, and smooth percentile-exceedance-frequency error. The third coordinate receives a larger weight, and exceedance indicators are sigmoid-smoothed; its tail gains should therefore not be described as arising without any tail supervision. Both context variants and the unconditioned baseline use this loss, with 40 training epochs, batch size 64, learning rate 8×10⁻⁴, and validation-objective checkpoint selection.

Key Experimental Results¶

Main Results¶

QG uses an independent long test trajectory. The following fixed-configuration comparison is taken from Table 2; every metric is lower-is-better. Area error is the Wasserstein-1 distance between empirical exceedance-area-fraction distributions, while frequency error is the absolute difference in the probability that at least one grid cell exceeds the threshold. Barred diagnostics average over thresholds 1.0, 1.5, and 1.75; Avg. additionally averages over both layers.

Model	T_train	Ensemble setting	D_KL	L_log	ψ2 Ē_AOT	ψ2 Ē_freq	Avg. Ē_AOT	Avg. Ē_freq
STORN	1000	—	0.00393	5.3454	0.00501	0.10564	0.01195	0.11726
STORN	50	—	0.07079	11.970	0.00589	0.14660	0.01574	0.12118
FiLM-STORN	50	(σ,M)=(0.01,2)	0.03646	8.2959	0.00391	0.06874	0.01047	0.07199

With the same short training data, FiLM improves all tabulated metrics. Against STORN trained on 1000 time units, FiLM trained on 50 time units improves all four averaged high-threshold diagnostics but has worse KL and log-density errors. Thus “better results with one-twentieth of the high-resolution data” must be restricted to the reported averaged exceedance diagnostics, not extended to the global PDF or every tail error.

The two density diagnostics are also distinct. KL weights errors by reference probability and is typically more sensitive to the bulk; L_log integrates the absolute difference between log densities with a numerical floor, over the region where reference density exceeds an estimation cutoff. It is not the logarithm of ordinary L1 error. Appendix Table 11 uses a log(L1) heading; interpretation should follow the definition in Appendix E.1.

Ablation Study¶

The following selection from Table 7 fixes the low-dimensional configuration at T_train=50, σ=10⁻⁵, and M=4, with an independent test horizon of 500. Values are means ± standard errors over 5 training runs. The attention backbone, loss, and optimization settings remain identical; only conditioning and context type change.

Coordinate	Context type	D_KL	L_log	E99
z1	No-FiLM	2.336 ± 0.243	7.103 ± 0.444	0.0084 ± 0.0023
z1	mean/std/eigs	1.884 ± 0.025	6.237 ± 0.110	0.0056 ± 0.0015
z1	eigvec	2.048 ± 0.341	6.745 ± 0.592	0.0048 ± 0.0007
z2	No-FiLM	2.728 ± 0.064	5.755 ± 0.251	0.0156 ± 0.0017
z2	mean/std/eigs	2.430 ± 0.238	6.015 ± 0.455	0.0137 ± 0.0017
z2	eigvec	2.112 ± 0.107	4.993 ± 0.205	0.0112 ± 0.0018
z3	No-FiLM	1.226 ± 0.109	7.231 ± 0.228	0.0042 ± 0.0005
z3	mean/std/eigs	1.210 ± 0.159	6.962 ± 0.164	0.0063 ± 0.0012
z3	eigvec	1.451 ± 0.153	7.330 ± 0.176	0.0039 ± 0.0011

E99 is the absolute model/reference difference in exceedance probability above the reference absolute-value 99th-percentile threshold. mean/std/eigs improves z3 density errors but increases E99 from 0.0042 to 0.0063; eigvec improves that frequency error but worsens KL and L_log. mean/std/eigs also worsens z2 L_log. Table 7 therefore does not show uniform improvements across every context, coordinate, and diagnostic.

The 26.3% improvement in main-text Table 1 and Appendix Table 6 is a sweep-selected z3 E99 result at T_train=150, not the improvement of the fixed configuration above. It should not be combined with another diagnostic's best setting into a single purported model.

Key Findings¶

QG has a useful conditioning-strength regime, but more members are not always better. In Appendix Tables 10–11, at T=50 and σ=0.01, increasing N_add from 2 to 8 changes KL from 0.036460 to 0.053113 and L_log from 8.2959 to 15.228, the latter worse even than the short-data baseline's 11.970.
Large noise can also destroy the local proxy: at T=50 and N_add=2, σ=0.2 gives KL=0.74884 and L_log=35.336, both worse than the unconditioned short-data baseline. The sweeps do not support effectiveness at arbitrary noise levels.
Wins on high-threshold averages do not imply wins at every threshold. In Appendix Table 12, fixed FiLM improves every lower-layer column, but upper-layer event-frequency error at the highest threshold is 0.13165, worse than short-data STORN's 0.10090; the two-layer average is also worse, 0.06722 versus 0.05300.
Appendix D.6 broadly states that more training data improve models, but Appendix Tables 10–11 give STORN at T=200 KL=0.10011 and L_log=13.828, worse than T=50. The actual tables do not establish monotonic improvement.

Highlights & Insights¶

The main contribution changes conditioning information rather than inventing FiLM or replacing STORN. Local dynamical summaries provide a relatively clean incremental interface for existing coarse-correction pipelines.
Deformation-kernel analysis advances the explanation of ensemble usefulness beyond a stochastic-regularization intuition. The low-dimensional OTD comparison additionally checks burst timing rather than only showing a better final PDF.
The high-dimensional example acknowledges that explicitly recovering directions is impractical and uses amplitude statistics instead. This reduces context dimensionality but leaves a gap between directional theory and actual QG features that requires empirical validation.
Separating area distributions from event frequencies distinguishes errors in event extent from errors in occurrence. Area fraction alone, however, does not uniquely identify location, shape, or duration and cannot establish complete spatial geometry.

Limitations & Future Work¶

The theory requires finite time, small-noise moment scales, and regularity. The low-dimensional system additionally contains a near-origin singular term, so these theorems alone do not establish that the example satisfies global Lipschitz assumptions. Long-horizon statistical gains remain empirical.
Test context replaces ensemble spread with rolling coarse variability without a quantitative shift-error bound. Future work could compare training/test context distributions and study failures when the coarse model entirely loses sensitivity signatures.
Ensemble construction increases coarse-solver calls and summary computation. The appendix reports CPU training taking hours and ensemble overhead small relative to high-resolution reference generation, but the textual numbers do not establish that the whole end-to-end pipeline is cheaper; no exact time or memory values are inferred from Figure 8.
Evaluation covers only one controlled three-dimensional system and one topographic QG configuration. It does not establish transfer to different PDEs, topographies, parameters, or real climate risks; bulk statistics can remain under-corrected with short data.
Source ambiguities include M versus N_add member counts, an unexplained boundary case between an M=1 sweep and a sample-standard-deviation denominator M−1, and Tables 10–11 describing STORN as deterministic despite its stochastic method definition. These details are not silently reconciled.
The pooled-ensemble control receives qualitative discussion, but the assigned text contains no complete numerical comparison table to verify. Exact relative reductions against it are therefore not reported; the source checklist only promises code release upon acceptance and supplies no verifiable repository link for the new method.

vs non-intrusive coarse correction: Barthel Sorensen and colleagues use nudging for supervised alignment and probabilistic recurrent networks to correct long-horizon coarse statistics. This paper adds ensemble summaries and FiLM within that pipeline rather than replacing the solver or supervised task.
vs OTD modes: OTD uses tangent dynamics to describe time-varying growth directions. The ensemble proxy avoids explicit Jacobian/tangent integration but only approximates relevant geometry under the stated assumptions; QG's four-dimensional context contains no explicit directions.
vs EnKF and stochastic augmentation: EnKF estimates state uncertainty with ensembles and performs observational updates; this paper uses reference-nudged spread as dynamical context. Similar means and variances do not imply identical objectives or interpretations.
vs rare-event sampling: Importance sampling, splitting, and large-deviation methods primarily estimate tail probabilities or transition pathways; this method produces corrected long trajectories. They can complement one another, but unbiased tail-probability estimation is not proved here.
Follow-up direction: Parameter-matched nongeometric context, temporally shuffled ensemble context, and causal-proxy calibration could separate added capacity, generic regime labels, and genuine instability information. This is a research question proposed by the note, not an experiment already performed in the paper.

Rating¶

Novelty: 4/5; combines finite-time deformation analysis with an ensemble-conditioning interface, with novelty in the information source rather than the basic network.
Experimental Thoroughness: 3/5; includes fixed-configuration ablation and long-horizon QG diagnostics, but proxy shift, member-count boundary cases, and cross-system generalization need further validation.
Writing Quality: 3/5; motivation is clear and appendices are extensive, but several notational, formula, and summary-claim inconsistencies require careful checking.
Value: 4/5; offers a reusable idea for data-limited scientific simulation, with value bounded by measured tail gains and explicit conditions.