Skip to content

ELHINN: Unifying Dense Crowd Simulation Across Scales via Eulerian–Lagrangian Hydrodynamics

Conference: ECCV 2026
Paper: ECCV Official
Code: https://github.com/shanshan-zys/ELHINN
Area: Physics-informed & Simulation (physics)
Keywords: Dense Crowd Simulation, Eulerian–Lagrangian Duality, Physics-Informed Neural Network, KAN Residual Correction, Entrance-Aware Resampling

TL;DR

To bridge the fundamental disconnect between macroscopic collective coherence and microscopic individual realism in dense crowd simulation, ELHINN introduces a cross-scale hydrodynamics framework where an enhanced Eulerian velocity model (HINN++) guides a microscopic Lagrangian trajectory network (PINN) augmented with KAN residual correction and entrance-aware resampling.

Background & Motivation

Simulating dense crowds in complex public spaces such as transit hubs, stadiums, and evacuation routes is crucial for architectural design, hazard mitigation, and interactive graphics. However, high-density environments exhibit intense nonlinear dynamics and heterogeneous social interactions that challenge conventional simulation paradigms. Existing approaches remain deeply fractured across spatial scales: macroscopic (Eulerian) formulations view the crowd as a continuous fluid governed by Navier–Stokes dynamics, effectively reproducing large-scale collective motion patterns but entirely omitting individual trajectories, identities, and heading behaviors. Conversely, microscopic (Lagrangian) frameworks track discrete agents via rule-based heuristics or recurrent neural networks, but struggle in dense bottlenecks due to groundtruth trajectory scarcity and the inability to maintain systemic motion coherence.

This scale gap induces a severe dilemma: microscopic trajectory evolution over long horizons easily collapses into erratic oscillations or collective stagnation without global momentum cues, whereas macroscopic continuum fields fail to capture discrete particle collisions, boundary egress, and influx flux. Previous literature typically treated these two perspectives as isolated alternatives, or relied on purely data-driven fitting that suffers from severe exposure bias during long-term autoregressive rollouts.

The core angle of attack in this paper is to reverse the conventional computational fluid dynamic paradigm: rather than reconstructing macroscopic fields from discrete agent tracks, it leverages continuous Eulerian velocity fields as strong physical priors to steer Lagrangian agent updates. Core idea: build a unified cross-scale framework (ELHINN) where macroscopic velocity fields are evolved via HINN++ with KAN residual correction and no-slip boundary enforcement, and then queried as spatial-temporal priors by a microscopic PINN with entrance-aware resampling (EARS) to achieve both systemic consistency and individual realism.

Method

Overall Architecture

ELHINN couples macroscopic velocity evolution and microscopic trajectory refinement into an iterative autoregressive pipeline. Given the initial Eulerian velocity field \(\boldsymbol{u}_0\), initial Lagrangian agent coordinates \(\boldsymbol{x}_0\), and the static walkable environment mask \(\boldsymbol{A}\), the simulation progresses step by step. First, HINN++ combines parameterized hydrodynamic operators, a Kolmogorov–Arnold Network (KAN) residual correction term, and no-slip boundary conditions to evolve the next-step velocity field \(\hat{\boldsymbol{u}}_{t+1}\). Next, the microscopic module samples local velocity vectors at current agent locations through bilinear interpolation, maps them to agent motion offsets using a lightweight MLP, and enforces entrance-aware resampling (EARS) to dynamically manage population flux across walkable boundaries.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    In["Initial States<br/>Velocity u0 + Positions x0 + Walkable Area A"] --> HINN["HINN++ Velocity Evolution<br/>Learnable Operators λ·F + KAN Residual + No-Slip BC"]
    HINN --> Conv["ConvResNet Spatiotemporal Predictor<br/>Forecasts Next Eulerian Velocity Field u_t+1"]
    Conv --> VQ["Bilinear Velocity Querying<br/>Interpolates Local Flow u_t(x_t) at Agent Positions"]
    VQ --> PINN["PINN Trajectory Refinement<br/>MLP Motion Mapping + Kinematic Coordinate Update"]
    PINN --> EARS["Entrance-Aware Resampling (EARS)<br/>Out-of-Bounds Removal + Inflow Detection + Respawn"]
    EARS --> Out["Cross-Scale Outputs<br/>Velocity Sequence U + Trajectory Sequence X"]

Key Designs

1. HINN++ and KAN Residual Correction: Marrying Fluid Dynamics with Symbolic Non-linear Fitting To overcome the inflexibility of fixed partial differential equations and the physical unreliability of pure black-box regressors, HINN++ formulates a learnable hydrodynamic governing equation. It integrates five physical and social operator feature maps \(\boldsymbol{F}(\hat{\boldsymbol{u}}_t) = [f_{\text{con}}, f_{\text{vis}}, f_{\text{ali}}, f_{\text{nav}}, f_{\text{coh}}]^\top\) representing nonlinear convection, viscous internal friction, group velocity alignment, short-term navigation along local currents, and boundary cohesion, weighted by a learnable vector \(\boldsymbol{\lambda} \in \mathbb{R}^5\). To capture intricate, anomalous crowd phenomena beyond analytical operators, HINN++ incorporates a lightweight Kolmogorov–Arnold Network (KAN) for adaptive residual correction: $\(\boldsymbol{f}_{\mathrm{gov}}(\hat{\boldsymbol{u}}_t) = \boldsymbol{\lambda}^\top \boldsymbol{F}(\hat{\boldsymbol{u}}_t) + \mathrm{KAN}(\hat{\boldsymbol{u}}_t)\)$ Combining parameterized physical operators with KAN's learnable univariate spline activations ensures strong inductive bias while maintaining the expressive capacity needed for sudden stops and localized crowd turbulence.

2. Environmental No-Slip Boundary Conditions: Explicit Obstacle Enforcement In geometries with sharp bends, bottlenecks, or obstacles, unconstrained velocity evolution often exhibits boundary penetration and unnatural wall-slip artifacts. HINN++ explicitly translates the walkable area \(\boldsymbol{A}\) into an environmental boundary condition \(\boldsymbol{f}_{\mathrm{bou}}\). Adopting the fluid mechanical no-slip condition (stipulating zero relative velocity along static walls), a corrective decelerating field is applied along the geometric boundary \(\partial \boldsymbol{A}\): $\(\boldsymbol{f}_{\mathrm{bou}}(\hat{\boldsymbol{u}}_t, \boldsymbol{A}) = \mathbf{1}_{\partial \boldsymbol{A}} \odot \hat{\boldsymbol{u}}_t\)$ where \(\mathbf{1}_{\partial \boldsymbol{A}}\) is the boundary indicator mask. Concatenated with the current velocity field, walkable mask, and governing features, this term instructs the downstream ConvResNet to enforce strict spatial containment.

3. Prior-Guided PINN Refinement: Injecting Continuum Momentum into Discrete Agents To bypass the heavy reliance on dense, noise-prone individual trajectory annotations, the Lagrangian refinement module uses the evolved macroscopic field \(\hat{\boldsymbol{u}}_t\) as a temporal-spatial guide. For agent \(i\), the local continuous velocity is queried via bilinear interpolation at its current position \(\hat{\boldsymbol{u}}_t(\hat{\boldsymbol{x}}_t^i)\). A compact MLP predicts an intentional motion adjustment vector \(\hat{\boldsymbol{v}}_t^i = \mathrm{MLP}([\hat{\boldsymbol{u}}_t(\hat{\boldsymbol{x}}_t^i), \hat{\boldsymbol{x}}_t^i])\), and updates the tentative position via discrete kinematics \(\tilde{\boldsymbol{x}}_{t+1}^i = \hat{\boldsymbol{x}}_t^i + \Delta t \cdot \hat{\boldsymbol{v}}_t^i\). This mechanism tethers individual motion to global flow streamlines, curbing unphysical trajectory drift.

4. Entrance-Aware Resampling Strategy (EARS): Dynamic Influx-Egress Equilibrium Open-world pedestrian environments involve continuous entrances and exits. Without dynamic population management, agents leaving the walkable region would accumulate at obstacles or cause the simulated crowd to depopulate over time. EARS executes a three-stage flux balancing routine at each step: 1. Out-of-Bounds Removal: Agents whose preliminary positions violate the walkable domain (\(\tilde{\boldsymbol{x}}_{t+1}^i \notin \boldsymbol{A}\)) are deleted; 2. Dynamic Entrance Detection: Grid cells along boundary \(\partial \boldsymbol{A}\) where the local velocity satisfies the inward-flow criterion \(\hat{\boldsymbol{u}}_t(\boldsymbol{x}_b) \cdot \boldsymbol{n}(\boldsymbol{x}_b) > 0\) (with inward normal \(\boldsymbol{n}(\boldsymbol{x}_b)\)) are identified as active entry points; 3. Agent Reinitialization: New agents are spawned near identified entryways to replace departed individuals, preserving total population balance and realistic throughput.

Loss & Training

To mitigate exposure bias during long-term autoregressive generation, HINN++ is trained via Scheduled Sampling, gradually shifting from groundtruth velocities to predicted fields under a Smooth L1 criterion: \(\mathcal{L}_{\text{data}} = \mathrm{SmoothL1}(\boldsymbol{U}, \hat{\boldsymbol{U}})\). The microscopic PINN is optimized without manual trajectory labels using a composite physics loss: $\(\mathcal{L}_{\mathrm{trajectory}} = \mathbb{E}_{i,t} \left[ \ell_{\mathrm{vel}}(i,t) + \alpha \cdot \ell_{\mathrm{col}}(i,t) \right]\)$ Here, velocity consistency \(\ell_{\mathrm{vel}}(i,t) = \| (\hat{\boldsymbol{x}}_{t+1}^i - \hat{\boldsymbol{x}}_t^i) - \hat{\boldsymbol{u}}_t(\hat{\boldsymbol{x}}_t^i) \|_2\) penalizes deviation from the queried Eulerian flow field (bypassed during EARS reinitialization), and the collision-avoidance term \(\ell_{\mathrm{col}}(i,t) = \sum_{j \neq i} \max(0, d_{\mathrm{min}} - \| \hat{\boldsymbol{x}}_t^i - \hat{\boldsymbol{x}}_t^j \|_2)\) penalizes inter-agent distances below the safety threshold \(d_{\mathrm{min}}\).

Key Experimental Results

Main Results

Evaluation is performed on the Dense Crowd Flow Dataset (DCFD, 457 video sequences across six flow patterns) for macroscopic velocity modeling, and the MOT20 benchmark (4 dense sequences, 2,332 annotated trajectories) for microscopic trajectories. Metrics include Inception Score (IS), Fréchet Inception Distance (FID), Structural Similarity (SSIM), dynamic smoothness metrics (perpendicular deviation \(D_i\), speed change \(\Delta V_{i,t}\), angle change \(\Delta A_{i,t}\)), motion intensity metrics (travel distance \(L_{i,t}\), energy \(E_{i,t}\), steering collision energy \(\mathrm{steer}E_{i,t}\)), and trajectory accuracy metrics (JADE, JFDE).

Table 1: Macroscopic Velocity Evolution Comparison on DCFD (Source: Table 2 in paper)

Method Paradigm IS ↑ FID ↓ SSIM ↑ Takeaway
PDE-Net Data-driven PDE 1.7016 0.2924 0.5237 Poor spatial coverage in large-scale flow
NSFnet PINN baseline 1.0369 0.7326 0.3526 Non-convolutional architecture fails on spatial fields
PPNN Physics-preserving 1.7440 0.0275 0.4893 Lacks boundary enforcement; structure degrades over time
HINN Predefined operators 1.6789 0.0638 0.4782 Fixed operator weights miss subtle nonlinearities
Ours (HINN++) Operators + KAN + BC 1.7469 0.0278 0.6030 Achieves top SSIM of 0.6030 with stable long-term fidelity

Table 2: Microscopic Trajectory Refinement Comparison on MOT20 (Source: Table 3 in paper; reported as signed deviations from Groundtruth, smaller absolute difference is better)

Method \(D_i\) dev \(\Delta V_{i,t}\) dev \(\Delta A_{i,t}\) dev \(E_{i,t}\) dev \(\mathrm{steer}E_{i,t}\) dev JADE ↓ JFDE ↓
Groundtruth (GT) 136.963 0.124 24.072 184.208 162.003 - -
Boids (Rule-based) +1285.546 +0.295 -13.807 +1848.020 +51.735 63.013 75.423
SFM (Social Force) +415.432 -0.080 -20.552 +75.114 -4.517 13.247 19.607
Social-LSTM -119.141 -0.123 -22.024 -27.842 -5.828 11.571 17.189
STGAT -47.650 -0.121 -23.096 -25.190 -5.791 12.431 17.935
PCS (Physics-infused) -125.164 -0.119 -23.545 -1.852 -5.475 9.231 14.897
SPDiff (Diffusion) -22.083 +0.097 -22.981 +292.087 +270.146 9.850 14.715
Sora (World Model) -43.776 +0.994 +24.311 +5231.787 +8066.645 19.186 21.535
Kling -39.108 +0.408 +5.731 +254.418 +417.607 28.672 33.744
Runway -53.206 +0.473 +6.608 +268.137 +426.389 22.199 23.856
Ours (ELHINN) -42.895 +0.314 +14.885 +0.979 +47.424 8.274 12.836

Ablation Study

Table 3: Ablation of Macroscopic and Microscopic Components (Source: Table 4 & Table 5 in paper)

Module Variant Metric 1 Metric 2 Metric 3 Analysis
HINN++ w/o BC (no boundary cond.) FID: 0.0395 SSIM: 0.4837 IS: 1.7883 Severe wall penetration; SSIM plummets by 0.1193
w/o KAN (no residual term) FID: 0.0350 SSIM: 0.6019 IS: 1.7497 Misses complex curved streamlines around bends
with MLP (replace with MLP) FID: 0.0382 SSIM: 0.6026 IS: 1.7937 Standard MLP lacks spline expressiveness; higher FID
Full HINN++ (Ours) FID: 0.0278 SSIM: 0.6030 IS: 1.7469 BC and KAN synergize for structure and detail
PINN w/o \(\ell_{\mathrm{col}}\) (no collision loss) JADE: 8.293 JFDE: 12.962 \(E_{i,t}\) dev: +1.421 Agent interpenetration increases in dense clusters
w/o MLP (direct integration) JADE: 8.307 JFDE: 12.969 \(\Delta V_{i,t}\) dev: +0.317 Lacks flexible kinematic compensation for individual intent
w/o EARS (no resampling) JADE: 13.257 JFDE: 17.438 \(D_i\) dev: -92.478 Broken flux equilibrium collapses agents into central flow
Full PINN (Ours) JADE: 8.274 JFDE: 12.836 \(E_{i,t}\) dev: +0.979 Maintains long-horizon stability and displacement fidelity

Key Findings

  • Eulerian guidance eliminates trajectory collapse: Microscopic baselines without global momentum fields (Social-LSTM, STGAT) degenerate into near-static predictions, while video diffusion platforms (Sora, Kling) diverge with unphysical energy explosions (\(E_{i,t}\) deviation exceeding +5000). ELHINN achieves a JADE of 8.274 and JFDE of 12.836, outperforming the strongest physics baseline PCS (9.231 and 14.897) by 10.3% and 13.8% respectively.
  • EARS is indispensable for long-range simulation: Disabling EARS causes catastrophic degradation: JADE surges from 8.274 to 13.257 (+60.2%) and JFDE jumps from 12.836 to 17.438 (+35.8%). Without boundary ingress/egress handling, population decay alters the density distribution and causes severe trajectory collapse toward dominant flow channels.

Highlights & Insights

  • Inverted Cross-Scale Information Flow: Conventional multi-scale methods attempt the ill-posed task of upscaling discrete particle states into continuum fields. Reversing the flow—using smooth Eulerian velocity evolution as a low-dimensional guide for high-dimensional Lagrangian trajectories—offers a stable, physics-grounded shortcut.
  • Physics-Preserving Splines via KAN: Rather than replacing physical operators with an end-to-end neural network, ELHINN retains five classical hydrodynamic operators for foundational conservation and delegates residual compensation to KAN, preventing physical hallucination while capturing high-order social interactions.
  • Geometry-Informed Inflow Detection: By checking the inner product of local velocity vectors with inward boundary normal vectors \(\hat{\boldsymbol{u}}_t(\boldsymbol{x}_b) \cdot \boldsymbol{n}(\boldsymbol{x}_b) > 0\), EARS dynamically discovers active entrances without manual gate labeling.

Limitations & Future Work

  • Extreme Sparsity Assumption: The framework relies on fluid continuity assumptions in dense settings; for sparse or dispersed crowds where individual destination goals dominate fluid advection, Eulerian velocity guidance may become less informative.
  • Pairwise Collision Cost: While the MLP trajectory step is efficient, calculating quadratic Euclidean collision penalties \(\mathcal{O}(N^2)\) becomes a computational bottleneck when scaling to millions of individuals, necessitating spatial indexing like BVH or hashing grids.
  • Future Directions: Extending the formulation to heterogeneous multi-class crowds (e.g., pedestrians, wheelchairs, and cyclists), emergency panic hazards with variable smoke resistance, and conditioning on multi-camera surveillance video.
  • vs HINN [ACM MM 2024]: HINN introduced hydrodynamics operators for dense crowd fields but remained confined to the macroscopic Eulerian domain without agent tracking. ELHINN establishes the cross-scale bridge and augments velocity modeling with KAN residuals and boundary conditions.
  • vs PCS & SPDiff [SIGKDD 2022 / AAAI 2024]: Existing physics-infused trajectory models rely on local repulsive kernels without global continuum advection, leading to fragmented paths over extended horizons. ELHINN leverages global Eulerian flow to maintain systemic coherence.
  • vs Sora / Kling / Runway Video Simulators: Video foundation models lack explicit momentum conservation and physical boundary awareness, leading to camera drift and trajectory energy explosions orders of magnitude above real data. ELHINN demonstrates the necessity of explicit physics priors for high-fidelity simulation.

Rating

  • Novelty: ⭐⭐⭐⭐☆ (Ingenious inverted Eulerian-to-Lagrangian coupling for crowd simulation; effective combination of KAN with fluid operators)
  • Experimental Thoroughness: ⭐⭐⭐⭐⭐ (Evaluated across 11 quantitative metrics spanning macroscopic and microscopic fidelity against 13 baselines)
  • Writing Quality: ⭐⭐⭐⭐⭐ (Crisp mathematical formulation, clean figures, and thorough ablations)
  • Value: ⭐⭐⭐⭐☆ (Highly relevant for high-density safety engineering, evacuation planning, and physically plausible crowd graphics)