FDR-Occ: Factorized Dense Routing for Full-Spectrum 3D Occupancy Prediction¶

Conference: ECCV 2026
Paper: ECCV Official Link
Cache: ../paper_cache/ECCV2026/eccv-3765.txt
Area: Autonomous Driving
Keywords: 3D Occupancy Prediction / View Transformation / Dense Routing / Resolution-Context Decoupling / Robust Perception

TL;DR¶

Addressing the severe locality bottleneck and rigid calibration dependence of conventional ray-based view transformation, FDR-Occ introduces Factorized Dense Routing (FDR) and a Resolution-Context Decoupled Architecture, achieving sub-quadratic global reachability while maintaining extreme robustness even when physical camera extrinsics are withheld.

Background & Motivation¶

Vision-based 3D occupancy prediction serves as a pivotal foundational capability for autonomous driving and mobile robotics. The central challenge lies in the 2D-to-3D view transformation (VT), which dictates how perspective camera features are lifted into the unified ego-car 3D volumetric space. Historically, mainstream frameworks have relied heavily on explicit physical ray priors, encompassing active forward-projection models such as the Lift-Splat-Shoot (LSS) family as well as reactive backward-querying mechanisms driven by 3D-to-2D cross-attention.

Despite their widespread deployment, physical-prior-based pipelines are inherently constrained by a severe architectural limitation: the rigid ray-wise localization of receptive fields (the Locality Bottleneck). In existing paradigms, the 2D-to-3D routing matrix is artificially restricted to an extreme sparsity pattern tightly aligned with physical lines-of-sight. This induces acute "Restricted Reachability" during lifting: an individual 2D pixel can only broadcast features strictly along its calibrated physical ray, leaving massive 3D volumetric regions completely unreachable to direct image evidence. When observing spatially continuous structures that span multiple cameras or extend behind occlusions, the network splats them as isolated fragments and is forced to rely on heavy downstream 3D/BEV convolutions to diffuse information. Moreover, this rigid design relies on an fragile assumption of perfectly accurate camera extrinsics; slight calibration drift or chassis vibration causes severe projection distortion.

Mathematically, the feasible function family bounded by physical ray routing is merely a strictly constrained subset of the unconstrained global bipartite routing space. To break this locality bottleneck without suffering from the quadratic complexity explosion of dense bipartite matching, the core idea is to approximate dense 2D-to-3D view transformation via hierarchical tensor contractions as Factorized Dense Routing (FDR), and decouple the 3D space into a global topological anchor and high-resolution geometric planes to unify macroscopic context with precise surface boundaries.

Method¶

Overall Architecture¶

FDR-Occ introduces Factorized Dense Routing (FDR) and a Resolution-Context Decoupled Architecture to eliminate the tension between global receptive fields and computational feasibility. Given multi-view perspective features $\mathcal{F}_{2D} \in \mathbb{R}^{N \times C \times H \times W}$ from a 2D backbone, the pipeline splits into two orthogonal pathways: a Global Context Pathway driven by FDR that uses progressive tensor contractions to capture wide-range scene layouts and outputs a $Z$-compressed Holistic Context Anchor $\mathcal{V}_{global} \in \mathbb{R}^{C \times X \times Y \times 1}$; and a Local Resolution Pathway driven by explicit LSS that avoids spatial contraction to carve exact surface geometries $\mathcal{V}_{LSS} \in \mathbb{R}^{C \times X \times Y \times Z}$, pooled into tri-perspective orthogonal planes. The two representations are unified on the shared BEV plane before decoding dense 3D semantic voxel probabilities.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    In["Surround-view Input<br/>F_2D"] --> Split{"Decoupled Routing"}
    Split -->|"Global Context Flow (Pt > 1)"| FDR["Factorized Dense Routing<br/>Tensor Contractions St->1, Vt->V"]
    Split -->|"Local Geometry Flow (Pt = 1x1)"| LSS["Explicit Physical LSS<br/>Depth Splatting to V_LSS"]
    FDR --> Anchor["Global Context Anchor<br/>V_global (Z-compressed BEV)"]
    LSS --> TPV["Tri-Perspective Pooling<br/>P_xy, P_xz, P_yz"]
    Anchor & TPV --> Merge["Shared BEV Unification<br/>P_xy = P_xy + V_global"]
    Merge --> Dec["Multi-Plane Broadcast & MLP Decoder"]
    Dec --> Out["3D Semantic Occupancy<br/>O ∈ R^(B×X×Y×Z)"]

Key Designs¶

1. Factorized Dense Routing (FDR): Approximating Global Reachability via Hierarchical Tensor Contractions

An unconstrained bipartite routing matrix $\mathbf{W} \in \mathbb{R}^{V_T \times S_0}$ bridging $S_0 = HW$ 2D pixels and $V_T = XYZ$ 3D voxels entails an intractable complexity of $\mathcal{O}(S_0 V_T C)$. Drawing inspiration from how convolutional layers factorize dense connections, FDR decomposes $\mathbf{W}$ into a sequence of $T$ localized block-matrix operators: $$\mathbf{W} \approx \mathbf{M}^{(T)} \mathbf{M}^{(T-1)} \dots \mathbf{M}^{(1)}$$ At stage $t$, the 2D feature grid is partitioned into non-overlapping patches $P_t = \Delta_h^{(t)} \times \Delta_w^{(t)}$ and unfolded into $\tilde{\mathcal{X}}^{(t-1)}$. Each patch is routed to an expanded 3D micro-volume of size $K_t = \Delta_x^{(t)} \times \Delta_y^{(t)} \times \Delta_z^{(t)}$ via a dynamically generated routing matrix $\mathbf{A}^{(t)}$ through a batched tensor contraction: $$\mathcal{Y}^{(t)} = \tilde{\mathcal{X}}^{(t-1)} \otimes \mathbf{A}^{(t)}$$ A Fold operation then integrates the expanded $K_t$ tokens into the 3D volume axis. Because the 2D spatial dimensions progressively contract ($S_t \to 1$) while the 3D dimensions exponentially expand ($V_t \to V_T$), early stages incur negligible costs due to the unexpanded 3D volume ($V_t \ll V_T$), while later stages run rapidly over the heavily contracted 2D grid ($S_{t-1} \ll S_0$). Under a practical 3-stage setting ($T=3$), FDR achieves 100% full-volume reachability while requiring merely $\sim 1.4\%$ of the unconstrained dense computation.

2. Adaptive Geometric Context Routing: Soft Inductive Bias over Rigid Masking

To prevent the dynamic routing weights $\mathbf{A}^{(t)}$ from searching aimlessly in unconstrained space, FDR conditions routing on soft geometric context rather than hard geometric masks. At stage $t$, intermediate spatial features are concatenated with aligned geometric features $E_{geo}^{(t)}$: $$\mathbf{A}^{(t)} = \text{Softmax}\left(\text{Unfold}_{P_t}\left(g_\theta^{(t)}\left(\mathcal{X}^{(t-1)} \parallel E_{geo}^{(t)}\right)\right)\right)$$ Here, $E_{geo}^{(t)}$ incorporates local depth representations predicted by a lightweight depth head alongside Plücker ray coordinates $E_{ray} = \text{MLP}(d \parallel (o \times d))$. In contrast to classical LSS, which enforces a hard zero for routing outside the physical ray, this soft conditioning acts as a flexible prior. It guides the model with geometric hints while allowing features to route across rays to infer occluded and cross-camera structures.

3. Resolution-Context Decoupled Unification: Orthogonal Dual-Pathway Synergy

Due to mandatory spatial pooling ($P_t > 1$), dense routing is subject to an intrinsic Resolution-Context Trade-off: expanding contextual reachability inevitably blurs high-frequency spatial boundaries. Rather than forcing a single operator to compromise both attributes, the framework establishes two decoupled, orthogonal pathways: 1. Global Context Pathway: Driven by FDR with its final stage configured to compress the $Z$-axis, generating a gravity-aligned Holistic Context Anchor $\mathcal{V}_{global} \in \mathbb{R}^{C \times X \times Y \times 1}$ dedicated to macroscopic topological consistency; 2. Local Resolution Pathway: Driven by classical LSS with $P_t = 1 \times 1$, strictly maintaining exact 2D pixel alignment to extract precise local geometry $\mathcal{V}_{LSS}$, subsequently pooled into tri-perspective planes ($P_{xy}, P_{xz}, P_{yz}$). Recognizing that autonomous driving layouts predominantly reside on the ground plane, the two representations are unified by direct residual addition on the shared BEV plane: $$\tilde{\mathcal{P}}_{xy} = \mathcal{P}_{xy} + \mathcal{V}_{global}$$ This resolves the structural trade-off with zero dense 3D fusion overhead, simultaneously capturing unconstrained global context and sharp local geometric boundaries.

Loss & Training¶

The entire network is trained end-to-end under a multi-task objective: $$\mathcal{L} = \mathcal{L}_{3D} + \lambda_{depth} \mathcal{L}_{depth} + \lambda_{sem} \mathcal{L}_{sem}$$ where the 3D occupancy loss $\mathcal{L}_{3D}$ combines weighted Binary Cross-Entropy (BCE) and Dice loss to address voxel sparsity and class imbalance. $\mathcal{L}_{depth}$ provides explicit depth supervision for the LSS branch, and $\mathcal{L}_{sem}$ applies 2D semantic segmentation supervision to regularize multi-view feature representations.

Key Experimental Results¶

Main Results¶

Evaluated on Occ3D-nuScenes and Occ3D-Waymo benchmarks using a ResNet-50 backbone with image resolutions of $256 \times 704$ and $640 \times 960$, respectively.

On Occ3D-nuScenes validation set (single-frame and +1 historical frame settings):

Method	Backbone	Temporal	mIoU (%)	IoU (%)	Drivable Surface	Car	Truck
FB-Occ	ResNet-50	Single	35.7	66.5	77.6	46.6	33.7
BEVDetOcc	ResNet-50	Single	37.1	70.4	80.2	50.0	36.8
LightOcc-S	ResNet-50	Single	37.9	-	80.0	48.7	35.1
ProtoOcc	ResNet-50	Single	39.6	-	-	-	-
ALOcc	ResNet-50	Single	40.1	70.2	81.2	52.1	39.5
Ours (FDR-Occ)	ResNet-50	Single	41.2	71.0	81.6	53.0	39.9
COTR	ResNet-50	+1f	41.4	-	81.8	53.3	39.5
DHD-M	ResNet-50	+1f	41.5	-	83.1	52.9	40.2
Ours (+1f)	ResNet-50	+1f	42.2	72.1	82.6	53.8	41.3

On Occ3D-Waymo validation set (multi-frame temporal fusion setting):

Method	mIoU (%)	Vehicle	Pedestrian	Road	Walkable	Building
BEVFormer	24.58	36.06	21.00	79.11	57.04	23.98
CVT-Occ	27.37	41.00	23.93	79.12	59.09	29.40
ALOcc-3D	30.03	39.61	24.14	78.42	59.91	35.67
Ours (FDR-Occ)	31.25	42.74	27.52	81.21	64.46	39.98

Ablation Study & Robustness Analysis¶

1. Component and Unification Strategy Ablations (Occ3D-nuScenes lightweight single-frame)

Configuration	mIoU (%)	IoU (%)	Note
Default Baseline	38.1	67.1	Full FDR global flow + LSS geometric flow decoupled unification
w/o Global Pathway (FDR)	37.0	66.0	Performance drops -1.1% without global context reachability
w/o Soft Geometric Hint	37.7	66.7	Lacks depth and ray conditioning, reducing routing efficiency
Only BEV Representation	37.7	67.9	Omits vertical orthogonal planes, impairing vertical resolution
Adaptive Weighting	38.0	67.2	Complex spatial weighting offers no benefit over simple residual sum
Late 3D Fusion	37.7	66.9	Fusing after 3D encoder misses early joint representation learning

2. Calibration Robustness in Uncalibrated Scenarios (No camera parameters provided)

Method	mIoU (%)	IoU (%)	Robustness Analysis
BEVDetOcc	4.7	33.1	Collapses completely due to rigid reliance on exact ray geometry
ALOcc	9.0	43.5	Distorted ray projections degrade features across cameras
FB-Occ	12.0	54.5	Retains partial capacity via attention but suffers from local bias
Ours (FDR-Occ)	28.0	63.2	Internalizes multi-camera rig topology via unconstrained routing

Key Findings¶

Substantial Gains on Spatially Continuous Structures: Classes requiring broad topological reasoning experience marked improvements, such as drivable surface on nuScenes, and Road (+2.79%) and Walkable (+4.55%) on Waymo.
Stage Progression Smoothness: Contracting 2D features in an abrupt 2-stage schedule ($20 \to 10$) lowers mIoU to 37.4%, confirming that progressive 3-stage aggregation ($10 \to 5 \to 4$) is critical for smooth contextual transition.
Intrinsic Topology Internalization: In the uncalibrated scenario where explicit extrinsics are withheld, physical baselines collapse to 4.7%~12.0% mIoU, whereas FDR-Occ maintains a solid 28.0% mIoU, proving that global routing can deduce rig geometry from visual context alone.

Highlights & Insights¶

Formulating View Transformation as Generalized Bipartite Routing: The paper provides a rigorous theoretical foundation demonstrating that physical ray projection is merely a sparse, constrained subset of global dense routing.
Breaking Quadratic Complexity with Factorized Contractions: Inspired by convolutional factorization, FDR alternates 2D patch unfolding and 3D volume folding across stages, achieving a 100% global receptive field with only $\sim 1.4\%$ of dense theoretical computation.
Decoupled Architecture Respecting Fundamental Trade-offs: Rather than over-engineering a monolithic operator, the framework assigns global topological abstraction to FDR and micro geometric localization to LSS, achieving clean synergy via BEV residual unification.

Limitations & Future Work¶

Hardware Memory Bandwidth: While theoretical FLOPs are reduced dramatically, frequent patch unfolding and multi-stage tensor reshaping can lead to memory bandwidth bottlenecks on embedded accelerators.
Future Directions: Exploring 3D Gaussian primitives for coarse geometric guidance within the routing weights, as well as extending the factorized routing mechanism across temporal dimensions for end-to-end 4D occupancy flow.

vs LSS & BEVDetOcc: Standard LSS enforces 1D discrete projection along rays, leaving significant 3D regions unreachable and sensitive to calibration errors; FDR softens ray constraints into dynamic routing to achieve full-volume reachability.
vs TPVFormer & FB-Occ: Attention-based lifting suffers from high computational overhead and resolution degradation; FDR-Occ unifies a BEV topological anchor with orthogonal geometric planes to balance efficiency and spatial precision.

Rating¶

Novelty: ⭐⭐⭐⭐⭐ First systematic formulation of view transformation as factorized dense routing, providing strong theoretical and architectural value.
Experimental Thoroughness: ⭐⭐⭐⭐⭐ Comprehensive validation across two benchmarks, fine-grained ablations, and a compelling uncalibrated stress test.
Writing Quality: ⭐⭐⭐⭐⭐ Clean narrative structure, crisp mathematical formulation, and tight alignment between figures and text.
Value: ⭐⭐⭐⭐⭐ Demonstrates a viable path toward calibration-robust, geometry-flexible 3D perception for autonomous driving systems.