BEV-GS: Feed-forward Gaussian Splatting in Bird's-Eye-View for Road Reconstruction

Conference: ECCV2026
Paper: Official paper page · PDF
Code: https://github.com/IRMVLab/BEV-GS
Area: Autonomous Driving / Road Surface Reconstruction
Keywords: bird's-eye view, feed-forward Gaussian splatting, elevation estimation, geometry-texture decoupling, novel-view synthesis

TL;DR

BEV-GS replaces perspective-pixel Gaussian prediction with elevation and texture prediction on a regular BEV grid, achieving 1.73 cm average absolute elevation error, 28.36 dB novel-view PSNR, and 26.3 FPS parameter prediction on RSRD; single-frame refers to inference input, while training still uses elevation labels and stereo-image supervision.

Background & Motivation

For a vehicle, road reconstruction must do more than produce realistic pictures: it must identify bumps and depressions along the upcoming wheel paths. Predicting perspective depth and projecting it into a point cloud provides geometry, but lacks a uniform spatial organization and inherits dense nearby pixels versus sparse distant pixels. RoadBEV predicts elevation directly on a bird's-eye-view grid, which suits wheel-path profiles, but does not jointly recover road color and texture.

NeRF and Gaussian splatting are strong appearance representations, yet road mapping methods such as RoGS and EMIE-MAP rely on image sequences and per-scene optimization, limiting their use for online road preview. Feed-forward methods such as Splatter Image and Flash3D reduce test-time optimization requirements, but allocating Gaussians per perspective pixel still concentrates representation capacity nearby. A fast renderer alone cannot prevent distant geometry errors and sparse sampling from causing texture misalignment.

The paper exploits a property of its target domain: a local road patch ahead of the vehicle can be described by one elevation per horizontal grid location. It therefore does not need to learn arbitrary Gaussian center distributions for general scenes. Geometry first locates the surface, and texture then retrieves image evidence along that surface. Core idea: learn elevation and appearance independently in a unified BEV grid, align texture queries with predicted elevation, and turn the grid directly into a renderable Gaussian set.

Method

Overall Architecture

The input is one road RGB image; the outputs are a local elevation map ahead of the vehicle and grid Gaussians for novel-view synthesis. The predefined region is 1.9 m wide and 5.0 m long, targeting the road the vehicle will traverse rather than the entire street scene. View transformation requires camera projection geometry, and novel-view rendering additionally requires a target camera pose. Single-frame does not mean calibration-free.

The geometry branch queries perspective image features at anchors with different heights, forms a BEV representation through weighted fusion, and predicts elevation. An independent texture encoder queries features from the same image onto the recovered surface and predicts spherical harmonic coefficients. The renderer assembles grid coordinates, elevation, and these coefficients into Gaussians, projects them into the target camera, and synthesizes the image without per-scene position and density optimization.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    Input["Single RGB image<br/>Camera projection geometry"] --> Geometry["Elevation-aware<br/>geometry decoding"]
    Input --> Encoder["Independent<br/>texture encoder"]
    Geometry -->|Predicted elevation| Texture["Elevation-guided<br/>texture queries"]
    Encoder --> Texture
    Geometry -->|Grid centers| Gaussian["Regular-grid<br/>Gaussian rendering"]
    Texture -->|SH coefficients| Gaussian
    Pose["Target camera pose"] --> Gaussian
    Gaussian --> Output["Novel-view road image"]

Key Designs

1. Elevation-aware geometry decoding: select useful height features before separating global displacement from local variation

The geometry encoder is a simplified EfficientNet with pretrained weights, producing features at 1/4 input resolution. For each horizontal grid location, the model places anchors at multiple heights, projects their 3D positions into the image feature map, and samples with bilinear interpolation. The resulting voxel features retain the image evidence associated with each possible height. This directly queries evidence for the target road surface rather than first estimating full-image depth and converting it into road points.

RoadBEV concatenates height features along the channel dimension. BEV-GS instead applies stacked 3D convolutions to produce height scores, followed by a height-wise softmax and weighted summation. This preserves information from important heights without passing an expanded channel dimension through the remaining decoder. The following is a readable restatement of cached Eq. (1) using its accompanying prose: \(g\) denotes a horizontal grid location, \(a_{g,z}\) is its convolution-produced height score, and \(F_{\mathrm{vox}}\) contains sampled voxel features.

\[ F_{\mathrm{BEV}}(g)=\sum_{z=1}^{N_z^g}\frac{\exp(a_{g,z})}{\sum_{k=1}^{N_z^g}\exp(a_{g,k})}F_{\mathrm{vox}}(g,z). \]

The geometry decoder does not directly regress a single elevation map. Vehicle pitch can shift the entire road relative to the fixed camera reference plane, so the model separates a global reference elevation from a local offset map. Average pooling, an MLP, and Tanh produce the global value. The local branch predicts probabilities over elevation-offset bins and takes their height-weighted average. This yields continuous elevation values rather than selecting a discrete height class.

\[ \widehat H(g)=\widehat H_{\mathrm{ref}}+\sum_{b=1}^{N_b}e_b\,\operatorname{softmax}(\ell_g)_b. \]

Here \(e_b\) is an offset-bin elevation and \(\ell_g\) contains the grid's bin logits. This combines the textual explanations of cached Eqs. (2) and (3), rather than claiming a character-perfect recovery of damaged equations. Both the global reference and local offset are bounded by \([h_{\min}/2,h_{\max}/2]\), so their sum spans the full elevation interval. The split distinguishes global camera-related shifts from local structures such as potholes, although the geometry-replacement ablation changes several components together and cannot isolate this decomposition's contribution.

2. Elevation-guided texture queries: retrieve appearance features from the predicted road surface

The texture branch uses a separate simplified MobileNetV3 encoder, also producing features at 1/4 input resolution. Geometry must identify structure and elevation variation, whereas texture must preserve color and detail; a shared backbone can make the objectives interfere. Separate backbones do not eliminate task interaction: geometry still determines the image locations from which texture should be sampled through 3D projection, providing a more explicit connection than sharing all intermediate features.

Without road elevation, a BEV location needs texture queries at multiple heights and may mix signals from different image positions. BEV-GS uses predicted elevation to determine its 3D location and projects that point into the texture feature map, requiring only one query per grid location. A texture decoder and upsampling layer then predict denser spherical harmonic coefficients. The order is \(L=1\), giving 12 color coefficients per location under the \(3(L+1)^2\) channel definition. These encode direction-dependent RGB appearance rather than merely copying input colors onto the surface.

The important ordering is alignment before decoding: elevation does not only position the final Gaussians; it also changes where texture evidence is retrieved. Removing elevation guidance preserves the geometry metrics but substantially damages appearance quality, supporting an explanation based on texture correspondence rather than simply increasing Gaussian count.

3. Regular-grid Gaussian rendering: encode the road prior in the Gaussian spatial distribution

Gaussian centers combine regular horizontal grid coordinates with predicted elevation, while the texture branch supplies spherical harmonic colors. In the feed-forward configuration, scales are set to \([0.002,0.002,0.002]^T\), rotations are identity matrices, and opacity is initialized to 1.0, making the initial Gaussians isotropic. Unlike general Gaussian reconstruction, the network need not freely predict every geometric attribute per pixel or rely on COLMAP initialization, densification, and position optimization to cover the road.

The constraint gives nearby and distant regions uniform coverage in road coordinates, mitigating the problem that fewer distant pixels imply fewer distant Gaussians. It cannot recover detail absent from the distant image evidence and only covers the predefined road patch. Rendering projects Gaussians into the target image, sorts them by depth, and performs opacity compositing. Several symbols in cached Eqs. (5) through (8) are corrupted, so this note retains the mechanisms established by the prose without reconstructing uncertain covariance or compositing equations.

The paper also offers optional test-time optimization: prediction initializes the representation, and current-frame color supervision updates color, rotation, and opacity. This is not the default feed-forward path, and its quality improvements should not be paired with feed-forward speed claims. Each iteration takes 11 ms and convergence occurs around 60 iterations, implying approximately 660 ms of additional iteration time. That estimate is direct multiplication and excludes other overhead; the cache does not provide reliably readable final metric values.

A Worked Example

Consider an input image containing a pothole ahead of the vehicle. This is an illustration of the mechanism, not an additional reported test sample. At each of \(64\times164\) geometry grid locations, the geometry branch queries 20 height anchors, fuses the relevant height features, and predicts the depression using a global reference elevation and probabilities over 40 offset bins. The pothole bottom and surrounding flat road are therefore not forced onto the same horizontal plane.

The texture branch uses these different elevations to query image features from the pothole bottom, rim, and surrounding road, then passes decoded spherical harmonic colors to the grid Gaussians. Given a target pose after forward vehicle motion, the renderer produces a road-preview image; the target frame is not an inference input. If geometry places the pothole bottom too high, both texture querying and Gaussian projection become misaligned, so geometric accuracy influences novel-view quality through two stages.

Loss & Training

The geometry-loss prose specifies smoothed L1 over grid locations with valid elevation labels. Texture supervision combines L1 and SSIM with weight \(\lambda=0.5\), considering only pixels actually covered by splatting. Norms and operators in cached Eqs. (9) and (10) are incomplete, so this note does not present an uncertain expanded loss as the original equation or assume an undocumented gradient-detachment implementation. The paper explicitly permits separate branch training, with geometry connected to texture through surface initialization and feature queries.

Rendering supervision uses the left and right RSRD camera images, rather than monocular unsupervised signals alone. Training runs for 50 epochs with AdamW, batch size 8, a maximum learning rate of \(5\times10^{-4}\), and linear decay. Both branches use 128 feature channels. Geometry grid spacing is 3 cm, and elevation spans 20 cm above and below the reference plane. All experiments use an NVIDIA A100; the reported speed is not an on-vehicle hardware measurement.

One dimensional ambiguity matters for reproduction: the implementation section specifies a \(256\times656\) texture grid, expanding each geometry-grid axis by 4, while the method section writes the spherical harmonic output dimensions as \(2N_x^t\times2N_y^t\). The cache does not clarify which dimensions refer to before or after upsampling. Accordingly, this note does not infer the final Gaussian count or invent the interpolation used to transfer elevation to the denser texture grid. The code URL comes from the cached abstract and was not checked online.

Key Experimental Results

Main Results

RSRD is split into 926 training samples and 285 test samples, with \(528\times960\) input images. Geometry metrics are average absolute elevation error, AAE; RMSE; and the percentage of grids whose absolute error exceeds 5 mm. Lower is better for all three. The prose uses AEE once for average absolute error; this note consistently follows the tables' AAE notation. Perspective-depth baselines are converted into BEV elevation maps for evaluation, and the relevant geometry baselines are retrained on RSRD for 50 epochs.

The following selects representative methods from original Table 1. Its RoadBEV row is merged in the text extraction, but the values are cross-checked against the results prose and Table 4.

Method	AAE (cm), lower	RMSE (cm), lower	>5 mm (%), lower
iDisc	2.70	2.89	86.3
Depth Anything V2	2.30	2.48	83.2
RoadBEV	1.93	2.16	83.4
BEV-GS	1.73	1.94	80.2

Novel-view synthesis results below follow original Table 2. Higher PSNR and SSIM and lower LPIPS are better. Figure 5 restricts baseline visualizations to the same road area, so these results should not be interpreted as evidence of full-street reconstruction capability.

Method	PSNR (dB), higher	SSIM, higher	LPIPS, lower
Splatter Image	22.32	0.70	0.41
Flash3D	23.27	0.71	0.41
BEV-GS	28.36	0.77	0.14

Original Table 3 reports 8M parameters, 26.3 FPS prediction, and 2061 FPS rendering for BEV-GS. These are separate stages, not 2061 FPS end-to-end reconstruction. The corresponding figures are 56M, 18.9 FPS, and 412 FPS for Splatter Image, and 400M, 6.3 FPS, and 96 FPS for Flash3D. Geometry-only RoadBEV runs at 26.8 FPS, so BEV-GS is not faster than every geometry baseline; its advantage is supporting both elevation and appearance with a small model.

Ablation Study

The following reproduces the measurements from original Table 4. RoadBEV replacement means replacing the geometry branch. Removing elevation guidance means removing it from texture queries, not eliminating elevation prediction.

Configuration	AAE (cm), lower	RMSE (cm), lower	>5 mm (%), lower	PSNR (dB), higher	SSIM, higher	LPIPS, lower
Replace geometry branch with RoadBEV	1.93	2.16	83.4	26.75	0.58	0.20
Shared geometry-texture backbone	1.89	2.13	82.1	24.90	0.48	0.28
Texture queries without elevation guidance	1.73	1.94	80.2	25.00	0.48	0.26
Full model	1.73	1.94	80.2	28.36	0.77	0.14

Key Findings

• Against RoadBEV, AAE decreases from 1.93 to 1.73 cm, a 0.20 cm or approximately 10.4% reduction. RMSE decreases from 2.16 to 1.94 cm, approximately 10.2%. The historical RoadBEV result mentioned in the introduction is not this same-protocol baseline and must not replace 1.93 cm here.
• Against Flash3D, PSNR improves by 5.09 dB and LPIPS decreases by 0.27. Sharing the backbone and removing elevation guidance reduce PSNR by 3.46 dB and 3.36 dB relative to the full model, respectively, supporting both task decoupling and geometric alignment.
• Without elevation guidance, AAE remains 1.73 cm while SSIM falls from 0.77 to 0.48. This directly supports geometry-guided texture querying, but does not imply that every appearance gain comes from a reduction in elevation error.
• The paper divides the road longitudinally into 15 segments of approximately 33 cm each, reports better performance than baselines in every segment, and keeps maximum segment error within 2.5 cm. Error still increases with distance. Exact per-segment values are not reliably readable in the cache, so no curve measurements are invented.

Highlights & Insights

• The representation is itself a prior. A regular road grid allocates reconstruction capacity by ground area rather than perspective pixels. For locally single-valued surfaces, this is more direct than generating arbitrary Gaussians and subsequently regularizing their distribution.
• Decoupling does not remove interaction. Two encoders specialize in structure and appearance, while elevation establishes explicit spatial correspondence. A transferable lesson is to let geometry assist appearance through sampling coordinates rather than expecting shared features to solve alignment implicitly.
• Evaluate geometry and appearance together. Sharper rendering alone does not establish more accurate pothole elevation, and the paper measures both. Conversely, centimeter-scale elevation error does not guarantee texture alignment, as the no-guidance ablation demonstrates.

Limitations & Future Work

• Author-acknowledged data boundary: All validation data were collected in sunny conditions with favorable illumination, without adverse weather or low-light nighttime coverage. Extending to these conditions is an explicit future direction, not an already demonstrated capability.
• Scope identified in this note: The representation covers a local 1.9 m × 5.0 m road patch with elevation bounded by 20 cm above and below the reference. Results do not directly extend to city-scale mapping, vertical structures, or arbitrary multilayer scenes. Larger regions and multilayer representations require further experiments.
• Millimeter precision remains unresolved: Even the full model has 80.2% of grid errors above 5 mm. A 1.73 cm average error does not establish millimeter-accurate road measurement. Effects on suspension control, safety decisions, and ride comfort are not evaluated.
• Supervision and deployment limits: Single-frame inference depends on labeled elevation and stereo rendering supervision during training, and runtime is measured on an A100. Cross-device calibration, domain transfer, and edge latency require testing; this is neither an unsupervised method nor a demonstrated on-vehicle deployment.
• Reproduction and attribution limits: The geometry-replacement ablation does not isolate height weighting from reference-plane decomposition. Corrupted cached equations and ambiguous grid notation prevent confirmation of all implementation details. Readable numerical endpoints for test-time optimization are unavailable, and none are fabricated here.

Related Work & Insights

• RoadBEV: Also predicts road elevation directly in BEV. BEV-GS changes height-feature fusion and elevation parameterization and adds an elevation-guided texture branch. It extends road reconstruction to joint geometry and appearance rather than introducing BEV road reconstruction itself.
• Splatter Image / Flash3D: Close neighbors in single-image feed-forward reconstruction, while BEV-GS emphasizes uniformly placed road-coordinate Gaussians and independent texture features. Its road-specific prior also constrains general-scene transfer; local road metrics do not establish universally better reconstruction.
• RoGS / EMIE-MAP: Recover road structure and color using road representations, but depend on sequences and per-scene optimization. BEV-GS shifts the main learning burden to training for online preview, at the cost of dependence on the training distribution and a limited local region.
• Research directions from this note: Geometry uncertainty could inform elevation-guided queries so that an erroneous height does not force texture retrieval from the wrong pixel. Temporal consistency is another possibility. Neither proposed extension is a module or result reported by the paper.

Rating

• Novelty: 4/5. Combines a road-grid prior, elevation-guided queries, and feed-forward Gaussians into a coherent method, building on RoadBEV and existing GS frameworks.
• Experimental Thoroughness: 3/5. Covers geometry, appearance, runtime, and key ablations, but uses one sunny-weather dataset and lacks cross-domain and fine-grained geometry ablations.
• Writing Quality: 4/5. Motivations and the two-branch causal chain are clear; texture dimensions are ambiguous, while cached equation extraction separately limits this verification.
• Value: 4/5. Relevant to online road preview and local surface digitization, but reconstruction metrics should not be equated with demonstrated vehicle-control benefits.