Adaptive Neural Dynamics for Robust Geometric LiDAR-Inertial State Estimation on UAVs

Conference: ECCV 2026
Paper: ECCV official page
PDF: Full paper
Area: Autonomous Driving / UAV State Estimation
Keywords: LiDAR-inertial odometry, IMU bias, neural SDE, adaptive gating, uncertainty propagation

TL;DR

NeuroSDE-LIO retains the geometric estimation backbone of FAST-LIO2, adaptively combines linear bias priors with neural stochastic bias dynamics, and propagates their uncertainty into the filter, reducing UAV-Lab absolute pose error from 2.13 m to 1.34 m at a LiDAR processing rate of 19.2 Hz.

Background & Motivation

LiDAR-inertial odometry uses high-rate inertial measurements to propagate orientation, velocity, and position, then corrects accumulated error with less frequent geometric constraints from point clouds. This division of labor depends on a reasonable IMU bias model. Low-cost MEMS sensors on a UAV experience motor vibration, mechanical stress, and abrupt maneuvers, producing structured, time-varying disturbances that a simple random walk may not capture. Gyroscope bias is particularly consequential: an orientation error also corrupts gravity compensation, which subsequently affects velocity and position. A strong point-to-map registration backend does not remove this prediction-stage weakness.

Learning bias dynamics addresses part of the problem, but replacing a random walk with a deterministic neural ODE leaves two gaps. A predicted bias trajectory does not express how vibration-induced randomness changes with the motion regime, so the estimated mean can improve while the filter remains overconfident. Meanwhile, hovering provides little excitation for identifying inertial biases from pose residuals. Updating a large network online cannot create observability that the measurements do not contain. The challenge is therefore to connect expressive bias prediction with both conservative behavior under weak excitation and meaningful filter uncertainty.

The paper preserves geometric measurement updates and changes the bias dynamics and uncertainty interface instead of learning the entire trajectory estimator. Core idea: use motion-dependent gating to combine a stable linear prior with neural SDE bias dynamics, propagate the resulting stochasticity into geometric filtering, and restrict deployment-time adaptation to low-dimensional calibration parameters.

Method

Overall Architecture

The inputs are synchronized LiDAR point clouds and six-axis IMU measurements. The outputs are orientation, velocity, position, inertial biases, and state covariance. Recent motion determines the mixture between linear and neural bias evolution; the resulting dynamics drive inertial prediction and covariance propagation. When a point cloud arrives, geometric residuals correct the state and provide evidence for accepting or rejecting a small online calibration update.

The paper calls its estimation backbone an IEKF. The important distinction for understanding the contribution is between two interfaces: bias prediction changes the corrected IMU signal, whereas covariance propagation changes how strongly the filter trusts that prediction. Reading the method as only a learned IMU correction misses the second interface.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    Input["IMU measurements<br/>and current orientation"] --> Gate["Adaptive Hybrid Bias"]
    Gate --> SDE["Neural Stochastic<br/>Bias Dynamics"]
    SDE --> Cov["Differentiable<br/>Covariance Propagation"]
    Cov --> Filter["Geometric filter update<br/>state and covariance"]
    Lidar["LiDAR point cloud"] --> Filter
    Filter --> Adapt["Constrained<br/>Online Calibration"]
    Adapt -->|update linear branch and gate| Gate
    Filter --> Output["Pose, velocity, and biases"]

Key Designs

1. Adaptive Hybrid Bias: make weak excitation a reason to trust a conservative prior

The system computes a motion-complexity measure, \(C(t)\), over a sliding window typically lasting 2–5 seconds. The readable description says that it uses angular velocity and acceleration after gravity compensation with the current orientation estimate, with squared norms reflecting motion intensity. Equation (1) is damaged in the cached text extraction, so the exact combination of terms cannot be verified and is not reconstructed here. In particular, the description should not be mistaken for a fully specified statistical variance. What is clear is that the current orientation influences this measure through gravity compensation, coupling the gate to geometric state uncertainty.

Separate smooth sigmoid gates control the gyroscope and accelerometer branches. Low motion complexity favors the linear model; stronger excitation increases the contribution of the neural model. Gate parameters determine the transition sharpness and threshold. The complexity range is calibrated with short static and aggressive-motion segments for each UAV/IMU setup, rather than being a universal threshold. This is not an argument that a network lacks capacity to fit hovering. It is an observability argument: pose residuals under weak excitation do not sufficiently identify the bias, so a more conservative branch is useful even when a powerful network is available.

2. Neural Stochastic Bias Dynamics: represent both systematic evolution and random disturbance

Gyroscope and accelerometer biases have separate neural dynamics. The drift architecture uses the current bias, IMU measurements, and measurement derivatives, with motion context and residual learning to capture nonlinear changes. Its target is bias evolution rather than a final pose, so the prediction still enters the standard inertial kinematic equations. Each network has 14,243 parameters, keeping the learned component small enough for sustained onboard execution.

Equations (4) and (5) can be written together using a shared sensor index without introducing additional model terms:

\[ d b_t^i = \left[\alpha_i(t) f_i(b_t^i,\tilde u_t;\theta_i) + (1-\alpha_i(t))L_i\tilde u_t\right]dt + \Sigma_i^{1/2}dW_t^i,\qquad i\in\{g,a\}. \]

Here the linear matrix maps the six-axis measurement into a three-dimensional bias derivative, the neural branch supplies nonlinear drift, and the diffusion matrix and Wiener process describe stochastic disturbance. The controlled neural ODE baseline retains deterministic drift but disables the learned diffusion head. It therefore tests whether modeling randomness provides value beyond fitting the mean evolution. Although the paper emphasizes heteroscedastic noise, the readable cached text does not adequately specify the diffusion-head parameterization or how valid diffusion matrices are enforced. Those implementation details should not be invented from the high-level SDE description.

3. Differentiable Covariance Propagation: align the filter's confidence with its learned prediction model

Using a neural model for the predicted bias while keeping random-walk covariance propagation would make the estimated mean and uncertainty follow different models. The method instead uses continuous-time covariance dynamics to connect the error-state Jacobian, process noise, and learned bias dynamics. Neural derivatives with respect to inputs and parameters are obtained through automatic differentiation, while linear components are differentiated analytically. Discrete propagation uses a second-order transition approximation. This makes the stochastic bias model part of the filter's prediction of future error, rather than a correction applied outside the uncertainty calculation.

There is also an indirect path from orientation error to bias prediction: orientation changes gravity compensation, which changes motion complexity, which changes the mixture weight. The discussion around Equation (14) explicitly includes this coupling. Treating the gate as a fixed external input would omit it. Conceptually, the extended state includes network, linear-branch, and gating parameters, but the paper allows network and linear-parameter covariance to be handled in a decoupled optimization thread while biases remain in the core filter. It does not imply full joint covariance estimation over every network weight at each IMU sample.

4. Constrained Online Calibration: freeze the high-dimensional model and adapt only a small interface

Offline pretraining uses IMU bias-residual labels obtained through Vicon-aided Rauch–Tung–Striebel smoothing. Evaluation flights are held out from pretraining and hyperparameter selection. During deployment, the neural weights stay fixed; only the linear branch and gate slope and threshold parameters are updated within a short sliding window. Thus, “adaptive” should not be interpreted as retraining the entire neural SDE while flying.

The update uses LiDAR residuals and regularization against parameter drift. Gradients are clipped, and updates are skipped when residual or normalized innovation squared consistency checks fail. Neural gradients do not directly overwrite the filter's nominal state. Geometric filtering remains responsible for state correction, while online calibration changes the model used for subsequent inertial prediction. Rejecting an unreliable update is particularly important when low excitation or abnormal observations make the current residual a poor calibration signal.

A Worked Example

Consider a flight that begins in a hover, makes a rapid turn, and returns to steady motion. During the hover, weak recent excitation favors the linear branch. The neural branch is not removed; its influence is reduced when the measurements provide little evidence for identifying a more complex bias evolution.

During the turn, angular and gravity-compensated acceleration excitation increase, raising the neural branch's weight. The stochastic model predicts bias evolution while the covariance path updates confidence in that prediction. The next LiDAR scan supplies a geometric correction; only if the residual and innovation checks pass does the system adjust the linear and gating parameters. This sequence illustrates the mechanism rather than reporting an additional quantitative flight experiment.

Loss & Training

The paper describes four objective components: pose accuracy, parameter regularization, uncertainty calibration, and temporal smoothness. The pose term constrains rotation and translation. Parameter regularization limits drift away from reliable settings. The covariance objective combines a trace term with a negative log-determinant term to discourage excessive uncertainty without allowing covariance collapse. Smoothness penalizes rapidly changing bias estimates.

The overall training formulation must be distinguished from the deployment-time parameter set. Network-weight regularization still appears in the symbolic objective, but Section 3.5 explicitly states that deployed neural weights are frozen. Several loss expressions are damaged in the text extraction, and the readable account does not provide verifiable optimizer, learning-rate, or training-epoch settings. No exact loss weights or missing hyperparameters are inferred here.

Key Experimental Results

Main Results

The UAV platform combines a Livox Mid-360 LiDAR, BMI088 IMU, and NVIDIA Orin NX. UAV-Lab uses a 32-camera Vicon system for reference poses. UrbanLoco and KITTI provide driving-domain evaluations; the paper explicitly treats KITTI as a structured-road sanity check, not the main evidence for aggressive UAV motion. APE is absolute pose error in meters. The reported RPE is an angular relative-pose measure in deg/m, not translational error. Lower is better for both.

Method	UrbanLoco APE (m)	KITTI APE (m)	UAV-Lab APE (m)	UAV-Lab RPE (deg/m)
FAST-LIO2	1.42	0.73	2.13	1.87
FAST-LIO2-NODE	1.15	0.61	1.78	1.52
NeuroSDE-LIO	0.89	0.46	1.34	1.12

Values are from Tables 1 and 3. The UAV-Lab APE reduction relative to FAST-LIO2 is approximately 37.1%. Among the newer LIO baselines in Table 2, iG-LIO reaches 1.48 m APE and 1.29 deg/m RPE, versus 1.34 m and 1.12 deg/m for NeuroSDE-LIO. This gives a more informative measure of the remaining gain than comparison with FAST-LIO2 alone. ORB-SLAM3 uses visual-inertial inputs, and cuVSLAM is only a hardware-aware VIO runtime reference; neither should be treated as a sensor-equivalent LIO comparison.

Ablation Study

UAV-Lab configuration	APE (m)	RPE (deg/m)	Change tested
FAST-LIO2	2.13	1.87	Original baseline
Base+Hybrid	1.76	1.55	Hybrid configuration without neural dynamics
Base+Hybrid+NODE	1.52	1.34	Deterministic neural drift without SDE diffusion
Full w/o Hybrid	1.63	1.42	Adaptive gate removed
Full NeuroSDE-LIO	1.34	1.12	Drift, diffusion, and gating enabled

These are Table 5 values. Removing the gate increases APE by 0.29 m; the hybrid NODE configuration without diffusion is 0.18 m worse than the full model. They support distinct contributions. The main-table FAST-LIO2-NODE and the ablation's Base+Hybrid+NODE are different configurations, so their 1.78 m and 1.52 m results must not be interchanged. The table does not independently remove covariance propagation or online calibration, and therefore cannot establish a contribution ranking for every module.

Key Findings

Metric	FAST-LIO2	NeuroSDE-LIO	Interpretation
UAV-Lab NIS interval coverage	70.5%	91.8%	The nominal target is approximately 95%, not 100%
Processing time per LiDAR scan	45.2 ms	52.1 ms	Accuracy gains require additional computation
Memory footprint	312 MB	385 MB	Onboard cost reported in Table 6

Normalized innovation squared is the quadratic form of the innovation under its predicted covariance, conventionally \(r^T S^{-1}r\). The reported coverage measures the fraction of values inside the corresponding 95% confidence interval. Agreement with nominal coverage is more informative than simply shrinking covariance. However, the main text does not sufficiently specify innovation dimensions and a unified cross-method computation protocol.

The full system processes LiDAR at 19.2 Hz while IMU preintegration and state propagation remain at 200 Hz. These rates refer to different parts of the pipeline. Without mixing, processing takes 51.6 ms per scan, suggesting that gating itself is inexpensive. On a held-out flight, gyroscope bias error falls from 0.00914 to 0.00363 rad/s, and accelerometer bias error from 0.05214 to 0.02070 m/s², providing evidence about the modified state variables in addition to trajectory accuracy.

One numerical inconsistency deserves explicit caution: the prose claims a 45.2% improvement in average RPE, whereas the displayed Table 3 averages, 1.28 and 0.74, imply approximately 42.2%. This note retains the tabulated measurements instead of repeating the conflicting aggregate claim. Additional decimal places inferred from rounded values would not resolve the discrepancy.

Highlights & Insights

• The learned interface is bias dynamics plus covariance, not the final trajectory. This preserves the ability of LiDAR geometry to correct localization while exploiting structured inertial disturbances.
• Gating is motivated by bias observability under weak excitation, not merely by computational cost. Freezing the large model and adapting a small calibration interface creates a more constrained deployment procedure than unrestricted online learning.
• Evaluation includes trajectories, bias agreement, and innovation consistency. These test the downstream task, the state variable being improved, and the reported confidence, respectively, giving a more explanatory picture than APE alone.

Limitations & Future Work

• The authors acknowledge additional neural inference cost and difficulty with extreme sensor saturation. A richer bias model cannot reconstruct information lost when measurements exceed the observable correction range.
• Each UAV/IMU setup needs motion-threshold calibration, and pretraining labels depend on Vicon-aided smoothing. The experiments do not establish calibration-free transfer across sensors, airframes, or vibration regimes.
• Independent covariance-propagation and online-calibration ablations, statistical uncertainty across runs, diffusion-head details, and full training settings are insufficiently documented in the readable main text. These gaps limit reproduction and causal attribution.
• Several cached mathematical expressions are corrupted, and the stated mean-square stability result should not be confused with a closed-loop safety guarantee. Multi-IMU integration, cross-sensor transfer, and certifiable safety are proposed future directions rather than demonstrated capabilities.

Related Work & Insights

• Versus FAST-LIO2: the geometric registration framework is retained, while bias dynamics and their uncertainty propagation are modified. The lower error comes with higher latency and memory use.
• Versus FAST-LIO2-NODE: the controlled comparison uses the same backend, training loss, and bias-input interface, but disables stochastic diffusion. This isolates the question of whether deterministic bias evolution is sufficient more directly than comparing unrelated networks.
• Versus iG-LIO, SR-LIO, and Adaptive-LIO: these methods address geometric registration, motion handling, or environmental adaptation, while this paper focuses on stochastic inertial bias evolution. The shared LiDAR–IMU logs in Table 2 support its complementary value but do not show that improvements from different directions are mutually exclusive.

Rating

• Novelty: 4/5. Adaptive gating, stochastic bias learning, and geometric uncertainty propagation form a targeted combination for high-dynamic LIO.
• Experimental Thoroughness: 3/5. Recent baselines, NODE/gating ablations, and direct bias validation are useful, but some module-isolation and statistical details are missing.
• Writing Quality: 3/5. Frozen deployment weights are clearly distinguished from online calibration, but aggregate numerical inconsistencies and implementation gaps remain; damaged cached equations separately constrain verification.
• Value: 4/5. A practical reference for high-dynamic onboard localization with low-cost IMUs, without established cross-device generalization or safety guarantees.