HAD: Combining Hierarchical Diffusion with Metric-Decoupled RL for End-to-End Driving

Conference: ECCV2026
Paper: ECCV Paper
Area: Autonomous Driving
Keywords: hierarchical diffusion, trajectory expansion, metric decoupling, reinforcement learning, reward caching

Title note: the conference task record uses "End-to-End Driving," whereas the cached PDF body uses "End-to-End Planning"; the rest of the HAD title is identical. This note retains the task title and bases its method and results on that PDF body.

TL;DR

HAD filters driving intentions before generating and refining structured local trajectories, trains with per-metric reinforcement learning and offline reward retrieval, and reaches 88.6 EPDMS on NAVSIM v2; its camera-only HAD-L variant reaches 47.5 Route Completion on the HUGSIM public set.

Background & Motivation

End-to-end driving planners map cameras, LiDAR, and ego status to future trajectories, but they differ in how this output is represented. Methods such as Hydra-MDP score a fixed trajectory vocabulary, while DiffusionDrive generates trajectories around anchors to avoid being entirely restricted to that vocabulary. Enlarging the candidate space, however, leaves a single ranking stage responsible for intention, local geometry, and driving safety simultaneously.

Diffusion sampling also has a driving-specific weakness. Independent Gaussian perturbations at each waypoint can turn a smooth path into an irregular one, forcing the model to repair unrealistic shapes before evaluating their suitability. Meanwhile, imitation alone does not express every safety rule, and a single reward combining collision avoidance, road compliance, progress, and comfort obscures which criterion favors a candidate. Computing every reward through a simulator further slows training.

These problems suggest that decisions at different scales need not share the same search space or supervision complexity. Core Idea: use expert-related coarse supervision to identify a few driving intentions, explore their neighborhoods with structure-preserving trajectories, and reserve detailed safety rewards and metric-decoupled optimization for local refinement, using cached reward approximations to reduce training cost.

Method

Overall Architecture

HAD takes images, LiDAR, and ego status and outputs a future ego trajectory. Its environment encoder follows Transfuser: the image and LiDAR branches each use ResNet34, producing other-agent query vectors and an 8 × 8 BEV feature map. A two-layer MLP encodes trajectories, while the intention decoder and refinement decoder each contain one Transformer layer and share the scene conditions.

Inference follows intention filtering, structure-preserved trajectory expansion, and local trajectory refinement. During training, metric-decoupled optimization uses offline reward retrieval to supply component-wise feedback for local candidates; it is not an additional simulator call at inference time. The final output combines expert-distance scores, predicted safety metrics, and RL weights through softmax-weighted trajectory averaging rather than simply returning the highest-scoring candidate.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    sensors["Sensors and ego status<br/>Environment encoding"] --> intention["Driving Intention<br/>Filtering"]
    intention --> expansion["Structure-Preserved<br/>Trajectory Expansion"]
    expansion --> refinement["Local Trajectory<br/>Refinement"]
    refinement --> output["Score fusion<br/>Weighted trajectory output"]
    refinement -.-> optimization["Metric-Decoupled Optimization<br/>and Offline Reward Retrieval"]
    optimization -.->|Training feedback| refinement

Key Designs

1. Driving Intention Filtering: narrow the region requiring detailed comparison

The first stage defines 20 sparse trajectory anchors that roughly cover different driving intentions. During training, truncated diffusion still applies Gaussian noise to these anchors. Trajectory queries then interact with scene features to produce denoised trajectories and region-classification confidence scores. The classification target indicates whether a candidate belongs to the same driving subregion as the human demonstration, and the top 2 candidates become the intentions for subsequent exploration. HAD therefore does not remove Gaussian noise from the entire pipeline; it changes how the later local candidates are expanded.

The coarse stage only identifies intentions worth exploring instead of handling all fine-grained safety comparisons. Section 4.3 summarizes its selection supervision as related to distance from the expert trajectory, whereas Section 3.1 specifies region-classification scores. Both emphasize narrowing the search with expert-related signals before optimizing the complete set of safety metrics.

2. Structure-Preserved Trajectory Expansion: perturb whole-trajectory scale and direction

Each retained trajectory is converted from Cartesian to polar coordinates, followed by a shared radial scale and angular offset across all waypoints. Default radial coefficients are 0.92, 0.96, 1.0, 1.04, and 1.08; angular offsets are −6°, −3°, 0°, 3°, and 6°. Their combinations produce 25 candidates per intention, or 50 for 2 intentions. Following the textual definitions accompanying Eqs. (5)–(6), the core transformation is:

\[ \rho_t^{(u,v)}=\lambda_u\rho_t,\qquad \theta_t^{(u,v)}=\theta_t+\delta_v. \]

The candidates are then converted back to Cartesian coordinates. The important property is not merely smaller noise: every waypoint shares the same transformation parameters, avoiding the shape disruption caused by independent perturbations. Compared with scaling XY coordinates directly, angular offsets provide lateral exploration even when a near-straight trajectory has negligible lateral coordinates. Structure preservation is nevertheless not a proof of vehicle-dynamics feasibility or collision freedom.

3. Local Trajectory Refinement: adjust geometry and assess the same candidates

The local decoder re-encodes expanded trajectories and adjusts their coordinates using the same environment features. It also predicts three categories of scores: distance-related scores to the expert trajectory, NAVSIM safety metrics, and an RL logit for each metric. Safety predictions estimate how a candidate will perform, whereas RL logits determine its selection probability for a particular optimization criterion; these are distinct heads.

At inference time, safety metrics are combined into a log-domain score using penalty and average components, with RL component weights aggregated similarly. Distance, aggregated safety, and aggregated RL scores receive coefficients of 0.6, 0.05, and 0.01, respectively. A softmax over the fused scores weights the final trajectory average. This allows outputs beyond discrete candidates, but the paper provides no formal guarantee that averaging candidates preserves their safety.

4. Metric-Decoupled Optimization and Offline Reward Retrieval: learn criterion-specific preferences within local groups

MDPO treats candidates expanded from the same intention as one group. For each driving metric, a separate softmax over RL logits defines selection probabilities, while the corresponding exact or approximated rewards are independently standardized within that group. Training maximizes the weighted sum of selection probability times normalized reward across metrics, then averages over intention groups. Using the notation of Eqs. (12)–(14):

\[ p_j^{(m)}=\frac{\exp(\hat{s}_{\mathrm{lcl},j}^{(m\text{-}\mathrm{rl})})}{\sum_{i\in\mathcal{I}_k}\exp(\hat{s}_{\mathrm{lcl},i}^{(m\text{-}\mathrm{rl})})},\qquad \bar r_j^{(m)}=\frac{r_j^{(m)}-\operatorname{mean}_{i\in\mathcal{I}_k}r_i^{(m)}}{\operatorname{std}_{i\in\mathcal{I}_k}r_i^{(m)}+\epsilon},\qquad J=\frac{1}{K}\sum_{k=1}^{K}\sum_m\alpha^{(m)}\sum_{j\in\mathcal{I}_k}p_j^{(m)}\bar r_j^{(m)}. \]

Here, \(\mathcal{I}_k\) indexes candidates in intention group \(k\), \(m\) denotes a metric, and \(\alpha^{(m)}\) is a predefined weight. Decoupling concerns metric-specific probability heads and within-group reward normalization; the final training objective still aggregates metrics. Nor is this the usual PPO objective with clipped new-to-old policy ratios: the phrase policy optimization should not be interpreted as implying PPO.

Rewards come from a dense vocabulary of 8192 reference trajectories, separate from the 20 intention anchors. The method precomputes component-wise safety scores for reference trajectories in the corresponding scene. During training, it retrieves the geometrically nearest reference and uses its cached scores as approximate rewards for the current output. This avoids repeated simulator calls, but accuracy depends on vocabulary coverage and the distance measure: near a collision boundary, similar geometry need not imply similar safety scores.

A Worked Example

Consider an illustrative approach to a bend, not an additional experimental result. After scene-conditioned denoising of 20 anchors, the first stage retains 2 plausible intentions. Each receives a 5 × 5 combination of radial scales and angular offsets, yielding 50 local candidates.

The local decoder adjusts these trajectories and predicts expert-distance scores, safety metrics, and RL logits. During training, each group of 25 candidates retrieves cached rewards and compares relative quality separately for each metric. At inference time, this reward-retrieval training procedure is absent; predicted scores are fused to output a weighted trajectory. Table 5 also evaluates 7 × 7 expansion at inference, producing 98 candidates for 2 intentions rather than the default 50.

Loss & Training

The total loss is a weighted combination of perception, global intention, and local refinement terms. Perception includes 3D box regression, category classification, and BEV semantic segmentation. The global term supervises trajectory regression and region classification; the local term supervises expert distance and safety metrics while incorporating the objective of maximizing the MDPO reward.

HAD consumes images and LiDAR. HAD-L follows Latent Transfuser by replacing LiDAR inputs with learnable positional embeddings, making it camera-only. The body does not specify the full epoch schedule, optimizer configuration, or all loss coefficients, and refers readers to an appendix for additional loss details. The available cache ends with references and contains no appendix, so those settings are not inferred here.

Key Experimental Results

Main Results

The paper describes NAVSIM as an open-loop planning benchmark that uses simulation to assess predicted trajectories against driving rules. HUGSIM evaluates closed-loop behavior across more than 400 scenarios grouped into Easy, Medium, Hard, and Extreme. They are not interchangeable measures of closed-loop capability.

PDMS multiplies penalty metrics by a weighted average of the remaining metrics. In v1, penalties are No Collisions (NC) and Drivable Area Compliance (DAC), while the average includes Ego Progress (EP), Time-to-Collision (TTC), and Comfort (C). Version 2 adds direction, traffic-light, lane-keeping, and extended-comfort constraints and reports EPDMS. HUGSIM RC measures route completion, while HDS measures aggregate driving quality rather than success rate.

Source and setting	Comparator	Comparator result	Ours	Result	Absolute gain
Table 1, NAVSIM v1 PDMS	DiffusionDrive, camera + LiDAR	88.1	HAD, camera + LiDAR	90.2	+2.1
Table 2, NAVSIM v2 EPDMS	EvaDrive, camera-only	86.3	HAD, camera + LiDAR	88.6	+2.3
Table 2, NAVSIM v2 EPDMS	EvaDrive, camera-only	86.3	HAD-L, camera-only	88.5	+2.2
Table 3, HUGSIM public Overall RC	ZTRS	42.6	HAD-L	47.5	+4.9
Table 3, HUGSIM public Overall HDS	ZTRS	28.9	HAD-L	30.8	+1.9

Starred baselines in Table 3 use both public and private datasets and should not be mixed with the public-set comparison above. HAD-L achieves Extreme RC/HDS of 39.1/22.5 against ZTRS at 21.9/11.0. However, its Medium scores of 49.3/31.4 are below ZTRS at 50.9/34.2, so gains are not uniform across difficulty levels and metrics.

Ablation Study

All results below are NAVSIM v2 EPDMS as reported in the paper. Rows from different tables represent different ablation axes, not one sequential accumulation experiment.

Source	Config	EPDMS	Supported interpretation
Table 4	Expert distance only in the local stage	86.7	Imitation-related scoring alone is insufficient
Table 4	Add local safety scoring	87.3	Safety supervision improves performance
Table 4	Further add local RL	88.6	Comprehensive local optimization works best
Table 4	Also add global safety scoring and RL	87.2	Complex rewards at more stages need not help
Table 5	K=2, 5 × 5 at training and inference	88.5	Explicit result for the default density
Table 5	K=2, training 5 × 5, inference 7 × 7	88.6	Denser inference adds +0.1
Table 5	K=20, 5 × 5 at training and inference	79.8	Removing intention filtering hurts substantially
Table 6	Random Noise / XY Expand / Polar Expand	84.9 / 85.9 / 88.6	Polar expansion beats both alternatives
Table 7	Single-head PDMS reward / MDPO	87.8 / 88.6	Metric decoupling adds +0.8

Key Findings

• Table 5 distinguishes 50 from 98 inference candidates. The body does not fully establish how the stated default parameters map to every headline 88.6 result, so this distinction should be retained.
• Section 4.4 reports reward-acquisition latency per trajectory falling from 0.2449 s to 0.0042 s, and total training time from 64.4 h to 13.6 h. The latter is approximately 4.74 times faster, summarized by the authors as about 5 times.
• HAD does not lead every component in Table 2: its EP is 87.4 versus EvaDrive's 96.6. A higher aggregate score does not imply improvement in every driving capability.

Highlights & Insights

• Assigning complex rewards to local comparisons matters more than simply adding reward heads. Table 4 shows that also adding safety and RL to the coarse stage reduces performance, suggesting that supervision complexity should match the search scale.
• Polar expansion encodes a whole-trajectory geometric prior. Sampling explores direction and length instead of making the denoiser repair independent waypoint perturbations.
• Offline caching targets reward computation as a separate training bottleneck. This can transfer to planning problems with expensive rewards and a discretely coverable candidate space, provided reward-approximation error is also measured.

Limitations & Future Work

• The authors provide no dedicated limitations section; the points below are analysis of method assumptions and experimental boundaries, not attributed author statements.
• Nearest-neighbor rewards can miss abrupt changes in collisions and road-boundary violations. A useful extension would call the simulator selectively near safety boundaries instead of always reusing the nearest reference reward.
• Top-K filtering can discard rare but necessary maneuvers early. Table 5 establishes the advantage of small K only in the evaluated settings, not under arbitrary distribution shifts.
• Neither structure-preserved expansion nor weighted candidate averaging provides a dynamics or safety guarantee. Feasibility checks are especially relevant when averaging trajectories from different local intentions.
• Evidence is limited to NAVSIM and HUGSIM, without real-road deployment validation. The available efficiency analysis also does not clearly separate offline precomputation cost, cache size, and hardware conditions.

Related Work & Insights

• vs DiffusionDrive: both use anchors and diffusion, but HAD first identifies a few intentions and explores their neighborhoods with polar expansion. The camera-plus-LiDAR comparison on NAVSIM v1 improves PDMS by 2.1.
• vs DiffusionDriveV2: its RL uses within-anchor and across-anchor group optimization with simulated rewards. HAD emphasizes per-metric probability heads, local within-group normalization, and cached reward approximations, not simply a renamed GRPO variant.
• vs Hydra-MDP / ZTRS: trajectory scoring is a shared foundation. HAD retains a dense vocabulary as a reward reference while generating and refining outputs beyond that vocabulary, separating the discrete supervision space from continuous output trajectories.

Rating

• Novelty: 4/5. Hierarchical search, structured expansion, and metric-decoupled training work together, although their basic ingredients are not entirely new.
• Experimental Thoroughness: 4/5. Two benchmarks and several ablation axes are covered, but reward-approximation error and real-road evaluation are missing.
• Writing Quality: 3/5. The main argument is clear, while default versus best sampling settings and appendix training details require further verification.
• Value: 4/5. The approach is directly relevant to end-to-end driving systems balancing planning quality against reward-computation cost.