GrowFields: Compositional 4D Neural Fields for Topology-Changing Plant Growth¶

Conference: ECCV 2026
Paper: ECCV Official
Project Page: https://joaquin-gajardo.github.io/growfields
Area: 3D Vision
Keywords: 4D Neural Fields, Plant Growth Modeling, Topology Changes, Compositional Neural Fields, Organ-level Tracking

TL;DR¶

GrowFields decomposes longitudinal plant point cloud series into canonical organ coordinate frames and learns a shared continuous 4D SIREN velocity field conditioned on per-organ latent codes, achieving high-fidelity 4D reconstruction and leaf-tip tracking under topology-changing and asynchronous plant growth.

Background & Motivation¶

Quantifying plant growth dynamics from longitudinal sparse 3D observations is fundamental to modern phenotyping, precision agriculture, and plant sciences. However, plants exhibit intricate spatiotemporal developmental behaviors: individual organs (stems and leaves at different tiers) undergo highly anisotropic non-rigid expansion and bending, accompanied by frequent topological mutations as new organs emerge and old ones senesce. Compounding these difficulties, 3D scans in real-world agricultural or greenhouse conditions are temporally sparse (e.g., typically once per day), meaning newly developed biological tissues possess no temporal correspondences in prior acquisitions, fundamentally breaking the classical assumptions of point tracking and topological conservation in dynamic scene reconstruction.

General-purpose 4D reconstruction frameworks—such as temporal signed distance functions (SDFs), global neural deformation fields, or dynamic point flow models—rely heavily on assumptions of globally smooth motion and invariant topology. When new organs sprout, these global methods attempt to warp existing geometries into novel structures, causing severe tearing artifacts or degenerate solutions. Recent plant-specific frameworks such as GrowFlow employ backward-time deformation to mitigate geometric emergence, but still formulate dynamics as a single monolithic field, overlooking organ identity and asynchronous growth. Conversely, heuristic plant phenotyping pipelines based on skeletonization or pairwise registration rely on engineered heuristics that lack robustness across diverse species and fail to yield continuous temporal trajectories.

Resolving these challenges requires embracing the natural modularity of plant architecture: decomposing the organism into structural units, decoupling global rigid poses from intrinsic biological expansion, and sharing a common developmental prior across organs. Core idea: decompose plant dynamics into localized canonical coordinate systems per organ, and drive forward geometric evolution via a shared continuous neural velocity field conditioned on learnable per-organ latent codes, enabling continuous, topology-adaptive 4D reconstruction and organ tracking via forward Euler integration without requiring generative emergence prediction.

Method¶

Overall Architecture¶

GrowFields takes a time-ordered sequence of segmented 3D plant point clouds $\mathcal{X} = \{\mathbf{X}^{(t)}\}_{t=0}^T$ as input, where each point carries a semantic instance label corresponding to its organ. The overall pipeline operates across three primary stages: organ decomposition and PCA-based canonical alignment, latent-conditioned continuous velocity field integration, and multi-step Chamfer trajectory supervision followed by global inverse recomposition. Each organ instance is first normalized into a canonical coordinate frame with its base at the origin and its principal elongation axis aligned with the $Z$-axis. In this canonical space, a single shared sinusoidal representation network (SIREN) models continuous velocity, conditioned on an organ-specific auto-decoded latent code $\mathbf{z}_i$. Points from the initial observed scan are integrated forward in time via Euler steps within a Neural ODE formulation, and the predicted states are transformed back into global world space to assemble the complete 4D plant trajectory.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Segmented Longitudinal Point Clouds<br/>X(t) with organ instance labels"] --> B["Organ-Level Canonical Alignment<br/>PCA local frame and rigid pose decoupling"]
    B --> C["Shared Latent-Conditioned Velocity Field<br/>SIREN continuous field + auto-decoded zi"]
    C --> D["Multi-Step Chamfer Supervision & Global Recomposition<br/>Cross-time ODE rollout + local-to-world mapping"]
    D --> E["Full-Plant 4D Reconstruction & Trajectories<br/>Temporal interpolation/extrapolation & leaf tracking"]

Key Designs¶

1. Organ-Level Canonical Alignment: Decoupling Global Rigid Pose from Intrinsic Growth

During plant morphogenesis, leaf expansion and stem tilting introduce substantial global rigid motion. Optimizing a deformation field directly in the global coordinate space forces the network to simultaneously model large-scale spatial rotations and subtle biological surface expansions, causing optimization to stall in sub-optimal local minima. GrowFields introduces a PCA-based canonicalization inspired by plant physiological coordinate principles: at each timestep $t$, each organ $\mathbf{X}_i^{(t)}$ is translated so that its base sits at the origin and rotated such that its principal growth axis aligns with the global $Z$-axis, yielding per-frame rigid transformation pairs $\{\mathbf{R}_i^{(t)}, \mathbf{t}_i^{(t)}\}$. This eliminates extrinsic rigid movement and isolates intrinsic morphological deformation. For dense temporal interpolation at unobserved intermediate timestamps, rigid transformations are smoothly reconstructed via spherical linear interpolation (SLERP) on rotation quaternions and linear interpolation on translation vectors.

2. Shared Latent-Conditioned Velocity Field: Unifying Growth Dynamics While Encoding Organ Specifics

Plant organs emerge at different times and grow at distinct rates, making a monolithic network unable to capture asynchronous behaviors, while training separate networks per organ sacrifices cross-organ geometric priors and multiplies parameters. GrowFields adopts an auto-decoder formulation using a single continuous velocity field parameterized by a SIREN network $f_\theta$. For each organ $i$, a learnable latent vector $\mathbf{z}_i \in \mathbb{R}^d$ is assigned (initialized from a zero-mean Gaussian $\mathcal{N}(\mathbf{0}, \sigma_z^2\mathbf{I})$). The canonical 3D coordinates $\mathbf{x}$, normalized organ time $t \in [0, 1]$, and latent vector $\mathbf{z}_i$ are concatenated into $[\mathbf{x}; t; \mathbf{z}_i] \in \mathbb{R}^{4+d}$ to predict local instantaneous velocities: $$ \frac{d\mathbf{x}}{dt} = f_\theta(\mathbf{x}(t), t, \mathbf{z}_i) $$ Geometry is propagated forward across time steps via discrete Euler integration $\hat{\mathbf{X}}_i^{(t+\Delta t)} = \hat{\mathbf{X}}_i^{(t)} + f_\theta(\hat{\mathbf{X}}_i^{(t)}, t, \mathbf{z}_i)\Delta t$. The shared network $f_\theta$ captures general morphological velocity patterns across organs, while the compact latent code $\mathbf{z}_i$ encodes organ-specific identities and individual growth rates.

3. Multi-Step Chamfer Supervision & Global Recomposition: Long-Horizon Fidelity Without Point Correspondences

Because cell division and continuous surface growth preclude true point-to-point correspondences across temporal point cloud acquisitions, conventional optical flow or direct trajectory losses cannot be applied. GrowFields supervises deformation via bidirectional Chamfer distance (CD). In each training iteration, a mini-batch of organs $\mathcal{B}$ and source-target timestamp pairs $(s, k)$ ($k > s$) are sampled. Geometry from the initial frame $\mathbf{X}_i^{(s)}$ is integrated forward across multiple steps to timestamp $k$ and supervised by $\mathcal{L}_{\mathrm{CD}}^{(i)}(\hat{\mathbf{X}}_i^{(k)}, \mathbf{X}_i^{(k)})$. This multi-step formulation prevents accumulation of single-step integration errors and ensures long-range dynamical coherence. During full-plant synthesis, newly emerging organs are incorporated at their first observed frame $t_{\text{first}}$, propagated forward in canonical coordinates, and reprojected into the global coordinate frame: $$ \hat{\mathbf{X}}_{i,\text{global}}^{(t)} = \hat{\mathbf{X}}_i^{(t)} \mathbf{R}_i^{(t)\top} + \mathbf{t}_i^{(t)} $$ The full plant is synthesized as the union of all active reconstructed organs $\hat{\mathbf{X}}_{\text{global}}^{(t)} = \bigcup_i \hat{\mathbf{X}}_{i,\text{global}}^{(t)}$, bypassing the ill-posed challenge of hallucinating unseen organ emergence from scratch.

Loss & Training¶

The parameters $\theta$ of the shared neural field and all organ latent codes $\{\mathbf{z}_i\}_{i=1}^N$ are optimized jointly end-to-end. The total objective consists of the average multi-step bidirectional Chamfer loss across sampled organs and an $L_2$ regularization penalty on the latent codes: $$ \mathcal{L}{\text{total}} = \frac{1}{|\mathcal{B}|} \sum}} \mathcal{L{\mathrm{CD}}^{(i)} + \lambda}} \mathcal{L{\text{code}}, \quad \mathcal{L}}} = \frac{1}{N} \sum_{i=1}^N |\mathbf{zi|_2^2 $$ where the bidirectional Chamfer distance between predicted points $\hat{\mathbf{X}}$ and target points $\mathbf{X}$ is formulated as: $$ \mathcal{L}|}}(\hat{\mathbf{X}}, \mathbf{X}) = \frac{1}{|\hat{\mathbf{X}}|} \sum_{\mathbf{x} \in \hat{\mathbf{X}}} \min_{\mathbf{y} \in \mathbf{X}} |\mathbf{x} - \mathbf{y2^2 + \frac{1}{|\mathbf{X}|} \sum|_2^2 $$ Each organ point cloud is independently split into 50% training and 50% test subsets across all time steps, with all 234 annotated leaf tips strictly held out in the test set. Full trajectory rollouts are evaluated end-to-end from the first frame } \in \mathbf{X}} \min_{\mathbf{x} \in \hat{\mathbf{X}}} |\mathbf{x} - \mathbf{y$t_0$ at inference time.

Key Experimental Results¶

Main Results¶

Quantitative evaluations are conducted on the TrackPlant3D benchmark across four species (Maize, Sorghum, Tobacco, Tomato) over 8 challenging growth sequences, utilizing Chamfer Distance ($\text{CD}, \text{mm}^2$) for geometric fitting and End-Point Error ($\text{EPE}, \text{mm}$) on annotated leaf tips for tracking fidelity. GrowFields is evaluated against both monolithic baselines (NDF, NVFi, DSR, CanFields, DPF) and part-aware formulations (COAP, per-organ MLPs).

Category	Method	MaC2 (CD/EPE)	MaC3 (CD/EPE)	SoC2 (CD/EPE)	SoH2 (CD/EPE)	TbC1 (CD/EPE)	TbS3 (CD/EPE)	Tm1H3 (CD/EPE)	Tm1S1 (CD/EPE)	Mean (CD/EPE)
Full-Plant Baselines	NDF [65]	14496.5 / 73.6	1227.9 / 95.8	1313.4 / 54.9	303.0 / 54.3	290.2 / 14.0	135.6 / 17.9	14.1 / 10.4	52.7 / 22.0	2229.2 / 42.9
Full-Plant Baselines	NVFi [32]	2178.9 / 72.2	1495.3 / 97.4	873.0 / 56.3	161.8 / 56.0	103.0 / 15.1	86.0 / 18.2	32.6 / 9.4	137.3 / 21.9	633.5 / 43.3
Full-Plant Baselines	DSR [64]	481.3 / —	194.9 / —	186.4 / —	61.1 / —	29.9 / —	25.0 / —	13.1 / —	84.0 / —	134.5 / —
Full-Plant Baselines	CanFields [67]	2025.7 / 77.6	1774.8 / 98.9	611.4 / 56.4	267.6 / 59.5	102.1 / 16.6	67.3 / 21.1	23.0 / 10.4	53.5 / 14.4	615.7 / 44.4
Full-Plant Baselines	DPF [52]	8.17 / 23.18	5.00 / 5.30	1.21 / 5.78	0.92 / 57.22	0.36 / 7.58	0.46 / 16.21	0.17 / 1.93	3.47 / 3.97	2.47 / 15.15
Part-Aware Methods	COAP [42]	4.84 / 7.57	2.48 / 3.87	1.31 / 7.76	1.02 / 5.61	1.03 / 3.22	0.65 / 2.55	0.38 / 1.40	1.82 / 1.88	1.69 / 4.23
Part-Aware Methods	Ours (per-organ MLPs)	2.41 / 3.68	1.52 / 2.07	0.87 / 1.47	1.01 / 3.44	0.70 / 3.28	0.48 / 2.67	0.13 / 0.74	0.33 / 1.59	0.93 / 2.37
Part-Aware Methods	Ours (full)	2.51 / 2.38	1.43 / 2.01	0.69 / 1.34	0.74 / 0.97	0.59 / 1.66	0.45 / 1.30	0.12 / 0.80	0.32 / 1.34	0.86 / 1.47

Ablation Study¶

Ablation experiments systematically evaluate the influence of canonical alignment, latent conditioning, multi-step supervision, latent regularization, conditioning mechanisms, and real-world robustness to automated segmentation models.

Experimental Configuration	CD ($\text{mm}^2$) ↓	EPE (mm) ↓	Empirical Analysis & Behavioral Impact
Ours (full model)	0.86	1.47	Complete framework: canonical alignment + shared SIREN + concatenated $\mathbf{z}_i$ + multi-step loss
w/o canonicalisation	0.96	4.23	Severe tracking degradation (+188% EPE), proving canonical frames are vital for stable organ motion
w/o latent code	1.05	3.44	Highest reconstruction error (+22% CD, +134% EPE), proving unconditioned fields fail on asynchronous organs
w/o multi-step CD loss	0.91	1.89	Noticeable temporal drift, confirming that multi-step rollout constrains long-range trajectory smoothness
w/o $\mathcal{L}_{\text{code}}$	0.88	1.74	Unbounded latent code drift slightly degrades out-of-distribution tracking stability
w/ FiLM conditioning [51]	0.96	2.47	Feature-wise affine modulation provides weaker inductive bias than direct coordinate-latent concatenation
w/ model segmentation*	0.86	1.42	Evaluated using automated PSegNet + TrackPlant3D labels, confirming strong resilience to realistic noise

Key Findings¶

Canonical alignment is critical for tracking precision: Omitting canonical alignment triples leaf-tip tracking error (EPE surges from 1.47 mm to 4.23 mm), demonstrating that untangling extrinsic rigid transformations is indispensable for modeling fine non-rigid growth.
Shared dynamics outperform isolated per-organ models: The shared field conditioned on latent codes outperforms training separate MLPs for every organ (Mean CD 0.86 vs 0.93, EPE 1.47 vs 2.37 mm), establishing that cross-organ regularization transfers effective growth priors across the plant.
Robustness in temporal extrapolation: When evaluated on 3 held-out future timesteps (Table 4), GrowFields delivers a dramatic improvement over COAP across all sequences (Mean CD 149.63 vs 451.09, EPE 18.99 vs 31.39 mm), validating the physiological viability of continuous ODE integration.

Highlights & Insights¶

Hierarchical disentanglement of plant morphogenesis: Factoring plant dynamics into global rigid transformation sequences via PCA alongside local canonical continuous neural fields drastically simplifies the optimization landscape and resolves topological changes.
Auto-decoded latent vectors as organ "genetic codes": Instead of training complex per-organ point encoders, simply optimizing a low-dimensional learnable latent code $\mathbf{z}_i$ provides the exact degree of freedom needed to capture organ-specific growth speeds within a unified neural velocity field.
Leaf-tip trajectory benchmark: Introduces a benchmark protocol featuring 234 rigorously annotated leaf-tip landmarks across diverse species, bridging the gap between computer vision 4D metrics and biological phenotyping needs.

Limitations & Future Work¶

Geometric sparsity under rapid growth: Because points from initial scans are advected forward without resampling, rapidly expanding organs exhibit lower surface point density at later timestamps; future work should incorporate adaptive densification mechanisms inspired by 3D/4D Gaussian Splatting.
Absence of predictive emergence: GrowFields incorporates new organs only upon their first empirical observation; coupling the framework with L-systems or morphogenetic generative models could enable proactive emergence forecasting.
Underconstrained dynamics in ultra-short sequences: On sequences with five or fewer training observations (such as SoH2), temporal velocity gradients remain underconstrained, leading to larger extrapolation variances.

vs CanFields / NVFi / NDF: Monolithic 4D methods enforce continuous spatial diffeomorphisms across the entire scene, which fail catastrophically when new organs emerge; GrowFields handles topology change naturally by decomposing plants into independent canonical components.
vs DPF: Dynamic Point Fields achieve strong local fitting via chained per-transition MLPs, but lack unified cross-organ inductive priors; GrowFields enforces global temporal consistency and superior tracking via continuous ODE formulation.
vs COAP / NASA: Part-based neural implicit fields were originally developed for articulated humans or rigid robotic links with fixed kinematic chains; GrowFields generalizes compositional neural representations to deformable biological structures undergoing asynchronous expansion.

Rating¶

Novelty: ⭐⭐⭐⭐⭐ Elegant combination of canonical organ frames and latent-conditioned neural fields for topology-changing biological growth.
Experimental Thoroughness: ⭐⭐⭐⭐⭐ Comprehensive evaluation on 4 species, 8 plant sequences, annotated leaf-tip trajectories, ablations, and automated segmentation tests.
Writing Quality: ⭐⭐⭐⭐⭐ Clear mathematical formulations, insightful motivation, and transparent discussion of limitations.
Value: ⭐⭐⭐⭐⭐ Establishes a foundational paradigm for continuous 4D plant digital twins and organ-level dynamic phenotyping.