RainODE: Continuous-Time Precipitation Forecasting with Latent Neural ODEs

Conference: ECCV2026
Paper: ECCV Paper
Code: https://github.com/SeongYE/RainODE
Area: Time Series / Earth Science
Keywords: precipitation forecasting, continuous-time modeling, latent Neural ODEs, stochastic source modeling, radar sequences

TL;DR

RainODE learns continuous precipitation trajectories with a discrete-teacher-guided latent Neural ODE and refines intensity and detail through a Brownian Bridge, supporting unseen temporal queries while raising RAPID-60min CSI-M from SimVP's 0.141 to 0.220, without guaranteeing a corresponding reduction in pixel-wise error.

Background & Motivation

Radar precipitation forecasting usually maps a sequence of historical images to future images at fixed intervals; a common SEVIR setting predicts 12 frames over the next hour. This interface suits discretely sampled observations but ties a continuously evolving precipitation process to the training time grid. When denser forecasts are needed, autoregressive models repeatedly feed predictions back into themselves, potentially amplifying location and intensity errors. Multi-input multi-output models such as SimVP and Earthformer predict a whole sequence at once, but additional output frames change computational and representational requirements. Time-query-based implicit models can specify the requested lead time without necessarily constraining separate queries to follow one shared dynamical trajectory.

Making time continuous is not sufficient by itself: precipitation moves while also growing, decaying, splitting, or merging locally. The paper motivates its architecture through advection, diffusion or mixing, and source/sink processes, assigning relatively smooth large-scale transport to deterministic dynamics. If that same deterministic model must explain multiple possible local intensity changes, MSE training can average uncertain heavy-rainfall cores away. Temporal coherence and sharp detail are therefore related problems that a single deterministic trajectory does not automatically solve together.

RainODE first learns a latent trajectory that can be sampled at arbitrary times, then conditionally refines the resulting precipitation field stochastically. Continuity primarily comes from the ODE rather than post-processing interpolation between predicted images; a separate stochastic module restores detail. The authors also introduce RAPID to test this division of responsibilities across intervals in the same region, spanning motion-dominated settings and settings with stronger structural evolution. Core Idea: stabilize continuous latent dynamics with a discrete teacher, use a shared trajectory for large-scale precipitation motion, and model local intensity changes omitted by deterministic forecasts with a Brownian Bridge.

Method

Overall Architecture

The input is a historical radar image sequence, and the output is a precipitation field at each requested future time, rather than a fixed class set or a global rainfall total. A 2D encoder extracts spatial latent features from each frame, and a spatiotemporal Translator predicts future discrete latent states for Discrete Teacher Guidance. Endpoint-Conditioned Latent Dynamics initializes its ODE with the first predicted latent state and conditions on the predicted endpoint to produce a continuous latent trajectory. A shared 2D decoder maps both teacher states and ODE states into images; the latter then undergo Stochastic Source Modeling to obtain the final forecasts.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    Input["Historical radar sequence"] --> Encoder["2D spatial encoding"]
    Encoder --> Teacher["Discrete Teacher Guidance"]
    Teacher -->|Predicted initial state and endpoint| Dynamics["Endpoint-Conditioned<br/>Latent Dynamics"]
    Dynamics -->|RK4 integration and shared decoding| Source["Stochastic Source Modeling"]
    Source --> Output["Precipitation at requested times"]
    Truth["Future ground truth: training only"] -.->|Teacher reconstruction supervision| Teacher
    Teacher -.->|Consistency after decoding| Dynamics
    Truth -.->|Bridge target| Source

Solid arrows show the prediction data flow, while dashed arrows show training supervision; future ground truth is not a deployment input. There are two distinct kinds of endpoint: the ODE's conditioning endpoint is predicted by the Translator, whereas the stochastic bridge's training target endpoint is a ground-truth image. This distinction prevents a deployable conditional forecasting model from being mistaken for interpolation that requires the future to be known in advance.

Key Designs

1. Discrete Teacher Guidance: learn useful discrete forecasts before constraining the continuous trajectory

The 2D encoder supplies spatial representations, and the Translator advances the historical latent sequence to a future discrete time grid. Decoding its future latent states produces teacher forecasts that receive reconstruction supervision directly from the true future sequence. The teacher is an internal discrete branch, not an additional large externally pretrained model. The same decoder processes the ODE branch, so alignment occurs in observable precipitation-image space rather than only between latent vectors. The teacher supplies task-relevant temporal anchors, preventing smoothness alone from becoming the objective of continuous dynamics.

The consistency term brings images sampled from the ODE trajectory closer to teacher images, while reconstruction brings the teacher anchors closer to ground truth. This explains why the discrete Translator remains: it supports trajectory learning rather than merely adding a redundant output head at odds with continuous modeling. Systematic teacher errors can nevertheless propagate through consistency training; the teacher is not a guarantee of the correct future. The main text does not specify whether gradients are stopped through the teacher branch, so an exact gradient route or an additional freezing policy cannot be assumed. The Table 4 ablation adds Latent ODE to SimVP, measuring the continuous-modeling stage as a whole rather than isolating teacher guidance alone.

2. Endpoint-Conditioned Latent Dynamics: learn a rate of change instead of repeatedly predicting frames

The initial ODE state is the Translator's first predicted latent frame, not the latent representation of the final observed image. Within the normalized forecasting interval, the state derivative depends on the current latent state, the predicted endpoint, and continuous time. Endpoint conditioning thus provides context for the full prediction interval, while the local derivative determines how the initial forecast evolves. The central relationship in the paper's Eq. (3) is clearly identifiable as:

\[ \frac{dz(s)}{ds}=f_{\mathrm{ODE}}(z(s),z(S),s). \]

Here \(s\) is normalized continuous time, \(S\) denotes the end of the forecasting interval, and \(z(S)\) comes from the predicted terminal latent state. The authors integrate this derivative with fourth-order Runge–Kutta, or RK4, and pass states at requested times through the shared decoder. Adding query times therefore does not require retraining a temporal mapping for a different output length; the dynamical parameters can be reused. Numerical integration and image decoding still require computation, so arbitrary-time queries do not imply cost-free increases in temporal density.

Instead of feeding a newly generated radar image back into the forecasting network, the method evolves a state within one latent dynamical system. This can help preserve the direction of large-scale motion, but numerical integration and the learned dynamics can both remain inaccurate. Derivative consistency primarily characterizes the ODE-defined trajectory, not a strict physical guarantee about atmospheric motion or the final stochastic outputs. In particular, the method does not explicitly solve the governing precipitation PDE or separately predict and supervise a physical wind-velocity field. Advection, diffusion, and source/sink processes motivate the division of modeling responsibilities; latent channels should not be equated individually with those physical variables.

3. Stochastic Source Modeling: recover local intensity changes on top of a deterministic trajectory

The deterministic branch can maintain precipitation location while averaging uncertain fine structures into smooth fields. The stochastic source modeling module, SSM, uses Brownian Bridge conditional diffusion with the ODE forecast and the true precipitation field as training endpoints. Intermediate bridge distributions are centered on a linear mixture of those endpoints and include stochastic perturbations that vary with the bridge step. According to Eq. (6) and Fig. 2, the mixing weight is \(k/K\) and the variance is \(2k(K-k)/K^2\), where \(k\) is the bridge step and \(K\) is the total number of steps. Perturbations vanish at the endpoints, while intermediate states can explore intensity and texture variations not explained by the deterministic forecast.

Bridge steps and physical forecast time are different axes: the former controls generative refinement, whereas the latter is specified by the ODE variable \(s\). Training can use the true future field to construct the bridge; inference must condition on the ODE forecast and use the learned generation process to refine it. SSM neither requires the true future image at deployment nor completes forecasting by merely adding random noise. The authors treat local growth and decay, sub-grid variability, and mixing as stochastic corrections rather than claiming to identify separate physical source terms explicitly. The remaining implementation follows BBDM; the main text does not fully specify the sampling schedule, denoising network, or training objective, so unverified implementation equations are not supplied here.

SSM should be evaluated through both heavy-rainfall detection and structural metrics, not just visual sharpness. Table 4 shows improvements in CSI-219, LPIPS, and SSIM alongside worse RMSE, matching its role of restoring local structure rather than minimizing every pixel-wise error. Applications prioritizing low false-alarm rates or numerical intensity accuracy should therefore not assume that the full model always beats Latent ODE.

A Worked Example

In RAPID-60min, the model receives 6 radar images sampled at 60-minute intervals and predicts 6 future hourly timestamps. The Translator first produces 6 discrete future latent states; the first initializes the ODE and the terminal state provides conditioning. To obtain an image at +90 or +150 minutes, the model queries and decodes the corresponding state along the continuous trajectory after the first forecast hour, without first generating endpoint images for image interpolation. SSM then conditionally refines that ODE precipitation field to restore heavy-rainfall cores potentially averaged away by deterministic forecasting. Figure 5 illustrates denser queries after hourly training, and Fig. 6 evaluates at 10-minute intervals up to +6 hours, both without retraining RainODE. This validates temporal densification within the established prediction range, not accurate extrapolation arbitrarily beyond six hours.

Loss & Training

Stage I combines teacher-to-ground-truth MSE reconstruction with ODE-to-teacher MSE consistency, weighted by \(\alpha=0.1\). The following is a compact notation based on the prose description, not a character-by-character restoration of the damaged Eq. (5) extraction:

\[ \mathcal{L}_{\mathrm{Stage\ I}}= \operatorname{MSE}(\hat{Y}_{\mathrm{teacher}},Y) +\alpha\operatorname{MSE}(\hat{Y}_{\mathrm{ODE}},\hat{Y}_{\mathrm{teacher}}). \]

No third ODE-to-ground-truth supervision term is inserted, because that is not the two-objective formulation described in the paper. Stage II applies the SSM refinement described above; the paper does not fully detail all optimization and freezing decisions between stages. Experiments use one NVIDIA H200 with 140 GB of memory and AdamW with an initial learning rate of \(10^{-3}\). Cosine annealing uses \(T_{\max}=100\) and \(\eta_{\min}=10^{-6}\) over 100 training epochs. Batch size is 12 for RAPID and 4 for SEVIR, and the checkpoint with the highest validation CSI-M is selected. Baselines are trained from scratch in the same H200 environment using their public implementations; this standardizes the environment but does not guarantee independently optimal tuning for every baseline.

Key Experimental Results

Main Results

SEVIR uses 13 context frames to predict 12 future frames at \(384\times384\) resolution, with June 1, 2019 separating training and testing data. RAPID comes from Korean Peninsula radar products with native 5-minute temporal and 0.5 km spatial resolution, cropped and resampled to 2 km for experiments. Each sample contains 6 input and 6 future frames at \(224\times224\) resolution; 2022–2024 supplies training data and 2025 supplies testing data. Intervals of 10, 30, and 60 minutes correspond to prediction horizons of 1, 3, and 6 hours; table values evaluate sequences rather than only their final frames. Frames are selected when more than 10% of the domain exceeds 0.1 mm/h, or the 99th-percentile intensity exceeds 10 mm/h, so the dataset should not be treated as unfiltered all-weather sampling.

The following excerpt is from Table 2, page 9; all metrics use a 0–1 scale, and CSI-219 thresholds encoded intensity at \(219/255\), not 219 mm/h. CSI is hits divided by the sum of hits, misses, and false alarms; CSI-M is the reported mean CSI across thresholds, and P1 denotes no spatial pooling. P16 denotes CSI-M after \(16\times16\) pooling, which tolerates spatial displacement; RMSE measures pixel-wise error and FAR measures false alarms, both lower being better.

Dataset	Method	CSI-M P1 ↑	CSI-M P16 ↑	CSI-219 ↑	RMSE ↓	FAR ↓
SEVIR	Earthformer	0.426	0.435	0.134	0.057	0.284
SEVIR	RainODE	0.430	0.544	0.177	0.060	0.399
RAPID-10min	Latent ODE	0.443	0.417	0.197	0.067	0.279
RAPID-10min	RainODE	0.420	0.597	0.187	0.082	0.418
RAPID-30min	exPreCast	0.234	0.437	0.055	0.101	0.561
RAPID-30min	RainODE	0.273	0.458	0.080	0.105	0.556
RAPID-60min	SimVP	0.141	0.155	0.011	0.107	0.626
RAPID-60min	RainODE	0.220	0.358	0.056	0.113	0.596

RAPID-10min is particularly important for avoiding overgeneralization: the full model improves pooled scores but trails Latent ODE in CSI-M P1, CSI-219, RMSE, and FAR. SEVIR also exposes trade-offs: better strong-event detection does not erase RainODE's higher RMSE and FAR relative to Earthformer.

Ablation Study

This excerpt is from Table 4(a), page 15; its values correspond to the RAPID-60min setting in Table 2. Lower LPIPS indicates smaller perceptual differences and higher SSIM indicates greater structural similarity; neither replaces meteorological event-detection metrics.

Config	CSI-M ↑	CSI-16 ↑	CSI-219 ↑	RMSE ↓	FSS ↑	LPIPS ↓	SSIM ↑
SimVP	0.141	0.400	0.011	0.107	0.130	0.346	0.580
Latent ODE	0.152	0.404	0.002	0.101	0.137	0.376	0.588
RainODE	0.220	0.400	0.056	0.113	0.223	0.275	0.690

Adding the ODE reduces RMSE from 0.107 to 0.101 but lowers CSI-219 from 0.011 to 0.002, indicating that smooth dynamics alone do not preserve heavy-rainfall cores. Adding SSM raises CSI-219 to 0.056 and SSIM to 0.690 while changing RMSE to 0.113, exposing a concrete accuracy trade-off.

Computational analysis comes from Table 4(b), page 15; it measures Latent ODE, not full RainODE with SSM.

Model	Output Length	Parameters (M)	FLOPs (G)
Latent ODE	6	9.32	68.84
Latent ODE	36	9.32	104.97
SimVP	6	13.62	90.91
SimVP	36	15.25	129.78

Key Findings

• Continuous parameter sharing is not cost-free densification: increasing outputs from 6 to 36 frames retains 9.32 M Latent ODE parameters but raises FLOPs from 68.84 G to 104.97 G.
• Long-horizon forecasts still deteriorate: Table 3, page 11, shows RAPID-60min CSI-M declining from 0.298 at the first future timestamp to 0.144 at the 6th.
• Unseen-interval evaluation involves different protocols: in Fig. 6, page 13, RainODE is trained hourly and evaluated every 10 minutes, while baselines are trained every 10 minutes and autoregressively extended to six hours.
• The high-threshold results in Fig. 6 support better heavy-rainfall detection, but low-threshold CSI-16 is slightly lower; the cache supplies no pointwise curve values, so exact gains are not invented.

Highlights & Insights

• Separating when to sample from how the system evolves provides a more flexible temporal interface than a fixed output length. Forecast frequency can change while reusing the same dynamical model.
• The teacher anchors the task, the ODE supplies continuous evolution, and SSM provides local stochastic corrections; their roles are reflected in the ablation metrics. In particular, opposing changes in heavy-rainfall detection and RMSE show that stochastic refinement is not a universally lossless enhancement.
• RAPID also analyzes non-advective evolution through residual intensity changes after optical-flow alignment. The authors define \(\rho=D_L/(D_E+\epsilon)\), where \(D_L\) is non-advective change after warping and \(D_E\) is total change; Fig. 3 shows an overall increase with the interval, motivating explicit treatment of local evolution.

Limitations & Future Work

• The authors acknowledge that radar alone lacks sufficient atmospheric context for long-horizon forecasting, particularly precipitation emerging from initially clear conditions. Adding wind, humidity, and other variables is their future multimodal direction, not an already validated capability.
• t-SNE trajectories provide qualitative coherence evidence, not proof of dynamical conservation or strict continuity of the final stochastic sequence. The paper also does not report a strict trajectory-consistency guarantee independent of embedding visualizations.
• Full SSM sampling latency, probabilistic calibration, and ensemble coverage are not sufficiently developed in the main results. Table 4(b) should not be interpreted as the end-to-end deployment cost of the full generative system.
• Table 1 uses VIL in kg/m² for SEVIR and rainfall rate in mm/h for RAPID, which are different physical quantities; the prose nevertheless labels SEVIR's 13.11 as mm/h, conflicting with the table header and preventing a same-unit comparison of extreme intensities.
• The cached extraction damages the formatting of Eqs. (2), (4), (5), and (6); this note retains identifiable relationships and prose explanations without inventing PDE coefficients, a bridge sampler, or missing hyperparameters. The code link comes from the paper and was not tested for executability during this reading.

Related Work & Insights

• vs SimVP / Earthformer: these supply discrete spatiotemporal forecasting baselines, whereas RainODE organizes future states through shared continuous dynamics. Its strengths concern temporal querying and long-horizon structure, not universal dominance on short-horizon pixel metrics.
• vs ClimODE / WeatherODE: these also use Neural ODEs, while this paper focuses on open-domain, high-resolution radar precipitation and unseen temporal intervals rather than only large-scale meteorological variables. Additional stochastic refinement addresses local intensity changes that deterministic trajectories struggle to explain.
• vs BBDM / PreDiff / CasCast: BBDM supplies the bridge-diffusion implementation foundation, while PreDiff and CasCast are generative precipitation baselines; RainODE places generative refinement after continuous-trajectory decoding. Investigating temporal consistency before and after refinement is a reader-proposed direction, not a completed validation in this paper.
• vs temporal interpolation: interpolation uses adjacent observations or discrete forecasts as endpoints; RainODE learns a time derivative and then queries states. Its trajectory is nevertheless guided by predicted initial and terminal states, so it is not independent of discrete temporal anchors.

Rating

• Novelty: 4/5. Teacher-guided latent ODEs and stochastic bridge refinement address a clear continuous-forecasting problem, although the underlying components have established precedents.
• Experimental Thoroughness: 4/5. Two datasets, multiple intervals, and stage ablations are covered, but full generative cost and probabilistic calibration remain insufficiently documented.
• Writing Quality: 3/5. The main argument and ablations are clear, but inconsistent physical units and missing teacher-gradient and bridge details complicate reproducibility assessment.
• Value: 4/5. The division of responsibilities is useful for dense temporal querying and strong-precipitation structure recovery, with explicit limits in long-horizon prediction and pixel error.