TEX-Drive: Temporal Perception Meets Experience-Guided Mixture-of-Experts for End-to-End Autonomous Driving¶

Conference: ECCV 2026
Paper: ECCV Official Link
Area: Autonomous Driving
Keywords: End-to-End Autonomous Driving, Mixture-of-Experts, Temporal Perception, Key-Frame Selection, Experience-Guided Planning

TL;DR¶

Addressing the disjointed modeling of temporal perception and functional specialization in end-to-end autonomous driving, TEX-Drive integrates an explicit key-frame temporal perception unit with an experience-driven Mixture-of-Experts planner, achieving a state-of-the-art Driving Score of 68.22 and 40.62% Success Rate on the closed-loop Bench2Drive benchmark.

Background & Motivation¶

Human driving relies on two foundational cognitive pillars: experiential reasoning and functional specialization. Experienced human drivers constantly recall past driving scenarios, extract temporal regularities from dynamic traffic, and deploy specialized behavioral modes—such as cautious lane-following or decisive overtaking—to handle complex situations. In contrast, existing end-to-end autonomous driving (E2E-AD) systems typically model temporal perception and behavior specialization in isolation. On the temporal perception front, conventional models either rely on recurrent structures or treat all historical frames uniformly through concatenation or dense attention, introducing severe noise and high computational overhead while struggling to recognize critical inflection points like sudden braking or lane cut-ins. On the functional specialization front, emerging Mixture-of-Experts (MoE) planners depend on shallow gating networks that route experts based solely on instantaneous, short-term features, rendering expert assignments unstable and opaque across long-horizon maneuvers.

The fundamental tension stems from the mismatch between dynamic temporal dependencies and persistent expert specialization: action planning requires coherent temporal representations that spotlight critical moments, while MoE systems require persistent, context-grounded experience signals across time rather than jittery per-frame decisions. Without temporal grounding, MoE gating behaves as an uncontrollable black box; without expert feedback, temporal encoders lack guidance on which historical cues truly dictate downstream maneuvers.

TEX-Drive resolves this dilemma by bridging temporal perception and expert routing into a unified, mutually reinforcing pipeline. Core idea: couple an explicit, interpretable key-frame temporal perception unit with an experience-driven MoE planner modulated by long-term memory and navigation goals, ensuring temporally consistent and semantically grounded trajectory generation.

Method¶

Overall Architecture¶

TEX-Drive comprises three principal components: a multimodal perception encoder, a key-frame guided temporal perception unit (KTP), and an experience-driven MoE planning unit (E-MoE). The system first fuses multi-view camera representations and ego-state measurements into a joint feature vector, storing recent frame embeddings in a first-in-first-out (FIFO) historical buffer. Next, the KTP module evaluates candidates using cross-temporal relevance and temporal proximity bias, deterministically selecting informative key frames to produce a fused temporal representation. Finally, the E-MoE unit injects high-level navigation goals via FiLM modulation, routes active experts by computing cosine similarities against long-term memory vectors, and autoregressively generates future waypoints through expert trajectory decoders.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Multi-View Images + Ego State Inputs"] --> B["Multimodal Perception Encoding<br/>Cross-attention fusion for current feature Ft"]
    B --> C["Historical Feature Buffer<br/>FIFO maintains recent K1 frames Ht"]
    C --> D["Explicit Key-Frame Temporal Perception<br/>Relevance & temporal decay scores filter Top-K2 frames"]
    D --> E["Goal-Conditioned Feature Modulation<br/>FiLM injects navigation goals into temporal cues"]
    E --> F["Experience-Guided Expert Routing<br/>Similarity with expert memory Mi selects Top-K3 experts"]
    F --> G["Multi-Expert Autoregressive Trajectory Decoding<br/>Confidence-weighted aggregation & trajectory head output"]

Key Designs¶

1. Key-Frame Guided Temporal Perception (KTP): Explicit Filtering via Cross-Temporal Relevance and Proximity Bias To prevent redundant historical noise and opaque frame averaging, KTP introduces structured temporal filtering over a FIFO buffer $H_t = \{F_{t-K_1+1}, \dots, F_{t-1}\}$ of size $K_1$. For each historical frame $F_i$, cross-temporal matching logits are computed against current feature $F_t$: $$L_{t,i} = \frac{(W_q F_t)(W_k F_i)^\top}{\sqrt{d}}$$ Unlike standard intra-frame self-attention, Softmax normalization operates along the candidate frame dimension, capturing the relative significance of each historical frame before spatial token averaging yields a scalar routing score $s_{ir}$. Combining quality scores $s_{iq}$, relevance $s_{ir}$, and an exponential proximity term $\text{TemporalBias}(i) = \exp(-(K_1 - i))$, the unified score is defined as: $$s_i = s_{iq} + s_{ir} + \lambda \cdot \text{TemporalBias}(i)$$ The top-$K_2$ frames are deterministically selected and merged via an exponential decay weighting $w_i = \exp(-\lambda(1 - \text{TemporalBias}(i)))$ within a hierarchical fusion network $f_{\text{fusion}}$ to construct the enhanced temporal feature $\hat{F}_t$, making temporal reasoning explicit and traceable.

2. Goal-Conditioned FiLM Modulation: Aligning Temporal Features with Intent Driving behaviors differ drastically under identical environmental cues depending on navigation goals. To guide expert selection towards the desired route, TEX-Drive conditions the fused temporal feature $\hat{F}_t$ on goal embeddings $G_t$ using Feature-wise Linear Modulation (FiLM). A lightweight MLP projects $G_t$ into channel-wise scaling and shifting factors $[\gamma_i \parallel \beta_i] = f_{\text{FiLM}}(G_t)$, producing target-modulated representation: $$\tilde{Z}_t = \gamma_i \odot \hat{F}_t + \beta_i$$ This step ensures that the subsequent expert matching aligns not merely with passive road observations, but with proactive tactical maneuvers.

3. Experience Router with Top-1 Momentum Memory Updates: Sustained Functional Specialization To overcome the instability of shallow gating networks, each of the $N$ experts $\{E_1, \dots, E_N\}$ maintains an experience memory vector $M_i \in \mathbb{R}^{d_m}$ capturing its long-term behavioral tendencies. The router calculates normalized similarity scores between $\tilde{Z}_t$ and each expert memory $M_i$: $$s_{it} = \frac{(\tilde{Z}_t W_q) \cdot (M_i W_k)^\top}{\sqrt{d_m}}$$ Softmax normalization yields routing probabilities $R_{it}$ to activate the top-$K_3$ experts. Crucially, during inference, only the highest-ranked expert receives a momentum update: $$M_i \leftarrow (1 - \mu) M_i + \mu f_{\text{proj}}(\tilde{Z}_t)$$ This momentum rehearsal mechanism adapts the winning expert to current distributional shifts while preventing representation collapse and homogenization across unselected experts.

4. Scalar-Gated Memory Fusion and Autoregressive Trajectory Decoding: Domain-Consistent Planning Each activated expert $E_i$ fuses its memory vector $M_i$ with modulated environment representation $\tilde{Z}_t$ using a learned scalar gate $\alpha = \sigma(W_g [\tilde{Z}_t \parallel M_i] + b_g)$, yielding memory-enhanced state $H_i = \text{LN}(\tilde{Z}_t + \alpha(M_i - \tilde{Z}_t))$. A lightweight autoregressive Transformer decoder then predicts future waypoint offsets $\{y_{t+1}^i, \dots, y_{t+T}^i\}$ conditioned on $H_i$. Finally, the candidate trajectories from all active experts are aggregated according to their routing confidences and refined by a shared trajectory head, ensuring dynamic smoothness and behavioral safety.

Loss & Training¶

The framework is trained end-to-end. Evaluated on Bench2Drive, future prediction is configured for $T=4$ steps at 2 Hz. The FIFO buffer size is $K_1 = 20$ with a 5-frame sampling interval. Low-level tracking utilizes tuned PID parameters (longitudinal: $K_P=5.0, K_I=0.5, K_D=1.0$; lateral: $K_P=0.75, K_I=0.75, K_D=0.3$), restricting throttle and braking to 0.75 and 0.4. All training runs were conducted across 4 NVIDIA RTX A800 GPUs.

Key Experimental Results¶

Main Results¶

On the challenging Bench2Drive closed-loop benchmark (comprising 44 reactive scenarios under diverse weathers) and standard open-loop trajectory error metrics, TEX-Drive surpasses 11 state-of-the-art baselines. Remarkably, using camera-only inputs, it outperforms both LiDAR-equipped multimodal architectures (e.g., LeTFuser, EATNet) and contemporary MoE-based planners (e.g., GEMINUS).

Table 1: Open-loop and closed-loop performance comparison on Bench2Drive (from Table 1 in paper)

Method	Modality	Open-loop Avg. L2 (m) ↓	Closed-loop Driving Score ↑	Closed-loop Success Rate (%) ↑
TCP (2022)	C	1.70	40.70	15.00
LeTFuser (2023)	C+L	0.84	52.53	18.18
UniAD-Base (2023)	C	0.73	45.81	16.36
ThinkTwice (2023)	C+L	0.95	58.79	29.54
VAD (2023)	C	0.91	42.35	15.00
DriveAdapter (2023)	C	1.01	64.22	33.08
EATNet (2024)	C+L	1.12	42.97	15.91
DriveTransformer (2025)	C	0.62	63.46	35.01
SparseDrive (2025)	C	0.83	42.12	15.00
TTOG (2025)	C	0.74	45.23	16.36
GEMINUS (2025)	C	1.60	65.39	37.73
TEX-Drive (Ours)	C	0.60	68.22	40.62

Table 2: Multi-ability success rates on Bench2Drive across five interactive tasks (from Table 2 in paper)

Method	Merging (%) ↑	Overtaking (%) ↑	Em-Brake (%) ↑	Give Way (%) ↑	Traffic Sign (%) ↑	Mean Ability (%) ↑
TCP	16.18	20.00	20.00	10.00	6.99	14.63
ThinkTwice	27.38	18.42	35.82	50.00	54.23	37.17
DriveAdapter	28.82	26.38	48.76	50.00	56.43	42.08
DriveTransformer	17.57	35.00	48.36	40.00	52.10	38.60
GEMINUS	11.11	37.50	55.00	40.00	45.26	37.77
TEX-Drive (Ours)	32.22	37.50	50.00	50.00	55.77	45.09

Ablation Study¶

Component-wise ablations illustrate the strong synergy between Key-Frame Temporal Perception (KTP) and Experience-driven MoE (E-MoE), where TP indicates unfiltered historical concatenation and MoE denotes standard gating.

Table 3: Component ablation study on Bench2Drive (from Table 4 in paper)

KTP	E-MoE	Driving Score ↑	Success Rate (%) ↑	Multi-Ability (MA) ↑	Note
✗	✗	53.77	21.81	19.59	Vanilla base model without temporal or MoE units
✗ (Standard MoE)	✗	55.15	26.81	21.37	Base model + standard MoE
✗	✓	61.72	33.18	37.14	Base model + E-MoE
TP	✗	57.86	25.00	20.83	Base model + unfiltered temporal features
✓	✗	62.74	34.38	40.09	Base model + KTP unit
TP	Standard MoE	58.27	31.25	33.59	Naive combination of unfiltered TP and standard MoE
✓	✓	68.22	40.62	45.09	Full model (TEX-Drive)

Table 4: Real-world generalization results on the nuScenes dataset (from Table 5 in paper)

Method	L2 1s (m) ↓	L2 2s (m) ↓	L2 3s (m) ↓	Avg. L2 (m) ↓	Coll. 1s (%) ↓	Coll. 2s (%) ↓	Coll. 3s (%) ↓	Avg. Coll. (%) ↓	FPS ↑
ST-P3	1.33	2.11	2.90	2.11	0.23	0.62	1.27	0.71	2.7
UniAD	0.48	0.96	1.65	1.03	0.05	0.17	0.71	0.31	1.7
GenAD	0.36	0.83	1.55	0.91	0.06	0.23	1.00	0.43	2.4
VAD	0.41	0.70	1.05	0.72	0.07	0.17	0.41	0.22	7.6
DWM	0.43	0.77	1.20	0.80	0.10	0.21	0.48	0.26	0.3
TEX-Drive (Ours)	0.45	0.59	0.91	0.65	0.08	0.13	0.39	0.20	3.3

Key Findings¶

Synergistic Amplification Over Isolated Modules: Adding KTP or E-MoE in isolation improves the Driving Score to 62.74 and 61.72 respectively. However, combining them (Table 3) yields a striking jump to 68.22 (+9.95 over TP + MoE), showing that KTP provides noise-free phase indicators for routing, while E-MoE stabilizes temporal focus through memory guidance.
Enhanced Routing Stability and Persistence: Quantitative analysis of expert gating (Table 3 in paper) demonstrates that TEX-Drive reduces routing entropy from 0.78 to 0.49 (decisive expert allocation) and boosts temporal persistence from 1.9 to 4.8 consecutive steps (2.5× longer stability), completely preventing erratic expert jitter.
Superior Long-Horizon Resistance to Degradation: On nuScenes (Table 4), as the prediction horizon expands from 1s to 3s, UniAD degrades by 244% in L2 error (0.48m to 1.65m), whereas TEX-Drive exhibits only a 102% increase (0.45m to 0.91m) and maintains the lowest average collision rate (0.20%).

Highlights & Insights¶

Interpretable Key-Frame Temporal Selection: Replaces unweighted frame concatenation and black-box attention with an explicit cross-temporal score, allowing verification of which historical moments dictate maneuvers like emergency stops.
Top-1 Momentum Experience Rehearsal: Updating only the dominant expert during inference prevents expert homogenization, preserving specialized behavioral repertoires across distinct navigation regimes.
Cross-Domain Generalizability: The strategy of indexing domain-specific memory vectors via goal-modulated temporal tokens is broadly applicable to other robotics problems requiring long-horizon reasoning.

Limitations & Future Work¶

Closed-Loop Verification Confined to Simulation: While open-loop trajectory metrics were validated on nuScenes, closed-loop driving safety remains tested exclusively in the CARLA-based Bench2Drive environment.
Hyperparameter Sensitivity Trade-offs: Increasing active experts ($K_3 > 2$) or historical key-frames ($K_2 > 3$) degrades performance due to expert interference and increased inference latency.
Future Directions: Integrating multimodal vision-language models for commonsense world reasoning and designing hierarchical episodic memory for open-world continual learning.

vs DriveTransformer / ThinkTwice: Prior works rely on heavy cross-modal Transformer attention that uniformly attends to historical frames; TEX-Drive explicitly prunes redundant temporal frames, achieving better accuracy at higher runtime efficiency.
vs GEMINUS / MoSE: Previous MoE driving approaches rely on instantaneous features to gate skills, causing high-frequency expert oscillation; TEX-Drive grounds expert selection in long-term memory vectors and goal modulation, stabilizing decision persistence.

Rating¶

Novelty: ⭐⭐⭐⭐☆ (The synergy between explicit temporal key-frame filtering and experience memory MoE routing effectively resolves expert jitter)
Experimental Thoroughness: ⭐⭐⭐⭐⭐ (Comprehensive validation spanning Bench2Drive closed-loop tasks, nuScenes real-world benchmarks, and quantitative routing entropy)
Writing Quality: ⭐⭐⭐⭐⭐ (Well-structured paper with clear motivation, rigorous mathematical formulation, and thorough analysis)
Value: ⭐⭐⭐⭐☆ (Provides an actionable and interpretable paradigm for robust long-horizon end-to-end autonomous driving)