Driving like yourself: A Benchmark for Closed-Loop Personalized End-to-End Autonomous Driving¶

Conference: ECCV2026
Paper: ECCV Paper
Area: Autonomous Driving
Keywords: personalized driving, closed-loop evaluation, style vectors, distribution matching, trajectory-head fine-tuning

TL;DR¶

Person2Drive connects human-in-the-loop simulator collection, individual style-distribution evaluation, and style-guided trajectory-head fine-tuning, raising average individual style similarity across two routes from 0.6652 to 0.6800, although driving-competence evidence remains simulation-based and not every metric improves.

Background & Motivation¶

End-to-end autonomous driving often pools records from different people into a single training objective, producing an average driving behavior. Whether passengers feel that a vehicle drives like them also depends on following distance, stopping gaps, acceleration and braking patterns, and lane-change timing, not merely route completion. Two trajectories that both obey road constraints can reflect very different comfort preferences and personal habits. Early personalization methods primarily adjust modules such as car-following or merging and are difficult to insert directly into a unified end-to-end model. StyleDrive groups behavior into aggressive, normal, and conservative categories, which supports conditional modeling but cannot fully distinguish drivers within a category. Language prompts can express driving intentions, yet they are not equivalent to continuous behavioral records that supervise subtle control differences.

Studying individual style also requires separating personal differences from road conditions. Comparing people driving on different roads cannot establish whether a speed difference reflects personality, speed limits, traffic density, or road geometry. A single trajectory likewise cannot reveal whether a braking maneuver is a stable habit or an incidental event. Bench2Drive supports closed-loop evaluation but uses agent-generated trajectories; some human datasets contain identity labels but primarily address open-loop or isolated driving tasks. The paper therefore first needs a platform where the same person repeatedly drives the same route before testing whether that person's style is consistently more similar to itself.

CARLA and Bench2Drive infrastructure make controlled human collection and closed-loop execution feasible. Person2Drive treats personalization as coordinated data, measurement, and adaptation interfaces rather than merely selecting another driving label. Core Idea: characterize individual behavioral distributions using repeated human trajectories on shared routes, learn a differentiable scene-conditioned style representation, and use it to guide individual trajectory-head fine-tuning.

Method¶

Overall Architecture¶

The task is individual-level end-to-end autonomous driving: given environmental information and a person's driving history, produce a closed-loop driving model whose behavior resembles that person. Person2Drive names the dataset and benchmark, whereas PDSA names the Personal Driving Style Adaptation method; they are not a single network. The process comprises controlled individual collection, scenario-conditioned style measurement, a scene-aware reward model, and individual trajectory-head adaptation. The first two components provide comparable behavioral data and target representations; the latter two turn those representations into optimization signals. The base driving model is DiffusionDrive, trained on Bench2Drive to establish its original driving competence. The reward model takes a trajectory and bird's-eye-view (BEV) scene information and outputs a 10-dimensional style vector; individual adaptation updates only the base model's trajectory prediction head. At deployment, the adapted driving model generates trajectories for closed-loop simulator execution, while the reward model evaluates style rather than replacing the driving policy as a controller.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Drivers and CARLA"] --> B["Controlled Individual<br/>Collection"]
    B --> C["Scenario-Conditioned<br/>Style Measurement"]
    C -->|Style-label supervision| D["Scene-Aware<br/>Reward Model"]
    D -->|Differentiable style supervision| E["Individual Trajectory-Head<br/>Adaptation"]
    F["DiffusionDrive trained<br/>on Bench2Drive"] --> E
    B -->|Personal trajectories and scenes| E
    E --> G["Closed-loop driving<br/>and style evaluation"]
    D -.->|Evaluate style| G

Key Designs¶

1. Controlled Individual Collection: distinguish habits from incidental actions through repetition

The collection platform extends Bench2Drive in CARLA and uses a Logitech G29 steering wheel and pedals for human vehicle control. A fixed top-down navigation map and rear-view mirrors support situational awareness instead of relying entirely on an unnatural simulator viewpoint. Routes are approximately 1 km long, providing room for lane-changing, following, and cornering while limiting the workload of each collection session. They include intersections, roundabouts, lane reductions, merges and diverges, highway exits, and varying lane counts rather than isolated events alone. Routes use XML representations with town and configuration metadata to maintain consistent settings across drivers. Identity annotation, sensor synchronization, and style-indicator extraction are automated, so expanding collection primarily requires additional driving sessions.

Data comes from 50 drivers on the same 4 routes, with the first 2 repeated 8 times each and the remaining 2 repeated 2 times each. This repetition supports training and cross-scene evaluation while providing evidence about intra-driver consistency and inter-driver differences. The paper reports 55 hours, 2.3 million LiDAR frames, and 16.1 million synchronized images, with annotations sampled at 10 Hz. Sensors include 1 LiDAR, 6 cameras, 5 radars, 1 IMU and GNSS setup, 1 BEV camera, and an HD map. The data also includes 3D boxes, depth, semantic segmentation, and driver identities; these are platform capabilities, not a requirement that the adaptation network consume every sensor. Each driver is treated as an independent style domain rather than being assigned to one of three discrete labels in advance.

2. Scenario-Conditioned Style Measurement: compare behavioral distributions rather than single-trajectory errors

The authors first identify vehicle-control and behavioral statistics that best distinguish individuals. They compute each driver's mean indicator, apply population-median-related processing, rank indicators by their cross-driver standard deviation, and retain the top 10. An important source uncertainty remains: Equation (1) on pages 7–8 is corrupted in the extraction, and the prose describes median normalization while the visible definition subtracts the median. This note explains the robust statistical selection intent without presenting division by the median or a reconstructed equation as the authors' exact definition. Figure 4 on page 14 contains candidate indicators such as TTC, lane-change frequency, longitudinal and lateral acceleration, and jerk, but its extracted text does not reliably recover the complete top-10 ordering. The 10-dimensional vector should therefore be understood as selected behavioral statistics, not a fixed list invented in this note.

Road conditions also change these statistics: highway speeds should not be interpreted using urban-road expectations. The authors treat each route as a scenario and perform separate min–max normalization using the feature distributions of all drivers on that route. Comparison operates on two sets of trajectory style-vector samples, using maximum mean discrepancy (MMD) with an RBF kernel to measure distributional differences. Intuitively, MMD compares within-set kernel similarities with cross-set similarities without requiring pointwise correspondence between trajectories. Equation (2) on page 8 is also corrupted in the extraction, so its double-sum expression is not reconstructed here. The similarity transformation in Equation (3) is identifiable as:

\[ \operatorname{MMDSS}=\frac{1}{1+\operatorname{MMD}}. \]

A higher MMDSS indicates more similar behavioral distributions; it is not a collision rate, driving score, or trajectory-coordinate accuracy metric. The paper states a range of \([0,1)\), but the formula gives a similarity of 1 when MMD is 0; this is an inconsistency in the source's boundary description. The complementary metric is histogram-based symmetric KL, using 50 bins, independent min–max normalization to \([0,1]\), and \(10^{-8}\) smoothing. Lower KL indicates greater similarity, but its value depends on histogram construction and sample support; the paper also observes instability in within-aggressive-style comparisons.

3. Scene-Aware Reward Model: provide a differentiable path for style alignment

Behavioral statistics describe driving but are not convenient differentiable supervision for every predicted trajectory, so the authors train a style-vector predictor. Rather than a scalar good-or-bad reward, it predicts the 10-dimensional vector corresponding to the statistical labels. The network uses a lightweight MLP with 2 hidden layers of 256 units each and ReLU activations. A BEV map is encoded into a global scene vector, injected into every time step of the trajectory encoder, and temporally aggregated to predict style. This scene conditioning lets similar motion changes be interpreted in their road context instead of treating every acceleration or deceleration as a personal preference. The reward model learns from ground-truth trajectories and their style labels, with Figure 1 indicating MSE supervision at this stage.

Learning a style vector must be distinguished from fully understanding a driver: the model remains dependent on the previously selected target indicators. The authors argue that learned representations help generalize to unseen behaviors, but this does not establish coverage of every subjective comfort preference or driving motive. Table 2 tests whether predicted styles reproduce ground-truth distributional relationships within and across categories, not whether the model controls a vehicle safely. For example, in Lane Following, within-style MMDSS is 0.9427 for ground truth and 0.9511 for predictions, a difference of 0.0084. This agreement supports its use as an approximate style supervisor but does not alone establish successful individual closed-loop driving.

4. Individual Trajectory-Head Adaptation: preserve the driving base while adjusting behavior with personal data

The base model first learns to drive on Bench2Drive, with training described as combining historical trajectory sequences and environmental BEV features. Individual adaptation then uses the target driver's historical trajectories and passes model predictions through the reward model to obtain predicted style vectors. MSE aligns these vectors with corresponding ground-truth style vectors, emphasizing behavioral statistics such as acceleration, spacing, and lane-changing rather than only pointwise coordinates. The perception backbone remains frozen, and only the final trajectory prediction head is fine-tuned, limiting how personal data perturbs the original driving representation. Figure 1 also shows L1 supervision between predicted and personal ground-truth trajectories; the prose emphasizes style MSE without fully specifying combination weights. PDSA therefore should not be described as using no trajectory supervision, nor should a total loss with definite coefficients be inferred from the diagram.

DFT is the direct fine-tuning control, testing how much personal trajectories alone can help. PDSA-WB removes BEV features from the reward model to test the additional value of environmental context for style alignment. PDSA fine-tunes the trajectory head under scene-aware style supervision and then uses generated trajectories for closed-loop execution and distribution-level style comparison. Despite the word reward, the central update is differentiable style-loss-driven fine-tuning, not an online reinforcement learning procedure. Freezing the backbone is a design choice for reducing interference rather than a safety guarantee; driving score and success rate must still be evaluated separately.

Loss & Training¶

Training follows 3 stages: base driving model training, style reward model training, and personal style-guided fine-tuning. Stage 2 learns the mapping from trajectories and scenes to style vectors, and stage 3 uses that mapping to supervise the trajectory prediction head. StyleDrive adaptation uses the aggressive-driving subset, whereas Person2Drive personalizes from each driver's history; their results address different targets. The supplied main text does not explicitly provide full loss weights, learning rates, training epochs, or exact per-driver training/test splits, so these hyperparameters are not supplied here. Reproduction also requires checking the supplement or implementation for population statistics needed on new routes and the boundary between reward-model training and evaluation data.

Key Experimental Results¶

Main Results¶

The following excerpt from Table 4 on page 12 reports Person2Drive MMDSS after individual adaptation; higher is better. Benefiting drivers are those with a positive gain over their own baseline, not a driving success rate; an average gain does not mean every driver benefits.

Method	Route 1 MMDSS	Route 1 benefiting drivers	Route 2 MMDSS	Route 2 benefiting drivers	Average MMDSS
Baseline	0.6410	Not applicable	0.6894	Not applicable	0.6652
DFT	0.6425	82%	0.7095	70%	0.6760
PDSA-WB	0.6439	76%	0.7131	58%	0.6785
PDSA	0.6465	84%	0.7135	76%	0.6800

PDSA achieves an average absolute gain of 0.0148, with a larger improvement on Route 2 than Route 1; the authors report paired Wilcoxon tests with \(p<0.001\). Table 4 also reports per-driver gain statistics labeled as mean and variance; their precision differs from subtracting rounded route means, and variance should not be casually relabeled as standard deviation.

Ablation Study¶

DFT, PDSA-WB, and PDSA in the preceding table already form an ablation sequence from direct fine-tuning to context-free style supervision and the full method. Adding BEV improves the two routes over PDSA-WB by 0.0026 and 0.0004, respectively, indicating positive but numerically small incremental gains. The following excerpt from Table 8 on page 14 examines closed-loop driving competence under the same adaptation strategy; higher is better in all four metric columns.

Method	Person2Drive driving score	Person2Drive success rate	Bench2Drive driving score	Bench2Drive success rate
Baseline	58.22	98.9%	54.91	13.27%
PDSA-WB	67.45	98.6%	54.16	15.22%
PDSA	70.13	98.4%	55.69	15.45%

Person2Drive driving score rises by 11.91 while success rate falls by 0.5 percentage points; Bench2Drive score rises by 0.78 and success rate by 2.18 percentage points. These results support the absence of a major overall competence collapse, not improvement in every safety metric or a real-road safety guarantee.

The following excerpt from Table 7 on page 13 reports a human preference study with 50 participants, a different measure from the closed-loop success rates above.

Method	Route 1 preference	Route 2 preference	Participants
Baseline	24.0%	28.0%	50
PDSA	76.0%	72.0%	50

Key Findings¶

Figure 2(b), page 10: intra-person MMDSS is 0.941 versus 0.681 between people, with KL values of 0.023 and 0.470, supporting style-separability tests using repeated human collection.
Table 3, page 12: aggressive-subset fine-tuning raises the corresponding MMDSS from 0.8453 to 0.8605, but this StyleDrive category-level experiment does not replace individual-level evidence.
Table 5, pages 12–13: A/N/C model averages are 0.6156, 0.6125, and 0.6174 versus 0.6800 for personal adaptation, illustrating the limitations of coarse categories for individual preferences.
Table 6, page 13 separately reports 0.7055 for PDSA and 0.6811 for BC, but the text does not explain the aggregation difference from Table 4; 0.7055 must not replace the main-experiment average.
The textual analysis of Figures 5–6 on page 15 emphasizes lane-change timing and post-stop following gaps rather than merely driving faster; the cache does not provide directly verifiable image details.

Highlights & Insights¶

Repeated same-route collection makes similarity to the same person testable. Its value lies not only in scale but also in estimating natural within-person behavioral variation.
Style is modeled as a distribution rather than a unique trajectory. Geometrically different trajectories can share statistical habits, reducing the risk of mistaking incidental coordinate errors for personal differences.
The paper reports behavioral similarity, driving competence, and human preference together. These answer different questions and provide stronger evidence than optimizing MMDSS alone.

Limitations & Future Work¶

The data is human driving in a simulator, not real-road vehicle data; 50 drivers and 4 routes do not establish coverage of population diversity or long-tail traffic conditions.
The authors report 55 hours, 2.3 million LiDAR frames, and 10 Hz, but the main text does not explain the accounting behind these totals; they should not be converted into a supposedly verified unified scale.
The reward model and evaluation share the same style-indicator system, creating a risk of metric-focused adaptation that misses subjective preferences; this is a reader inference, and the preference study supplements but does not eliminate it.
The main text does not sufficiently detail blinding, order randomization, or question design in the preference study, so preference proportions should not be interpreted as causal evidence of long-term trust.
MMDSS kernel parameters, normalization boundaries, the exact form of Equation (1), and loss weights require further verification; full reproduction still depends on supplementary material or code.
Future studies could test few-shot adaptation to new drivers, cross-route and real-vehicle transfer, and explicit constraints against imitating unsafe personal habits; these are suggestions, not implemented features.

Compared with StyleDrive: StyleDrive provides three style categories and a semi-closed-loop setting, while Person2Drive emphasizes identity annotations, repeated individual trajectories, and individual closed-loop adaptation; the difference is not simply adding more labels.
Compared with Bench2Drive / DiffusionDrive: the former provides closed-loop infrastructure and base training data, and the latter supplies the driving base; the main contribution is individual-style collection, measurement, and adaptation interfaces rather than a new general perception backbone.
Compared with language-conditioned driving: language expresses intentions, whereas trajectory statistics constrain continuous motion habits, making them complementary; separating immediate instructions from long-term preferences is a reader-proposed extension.

Rating¶

Novelty: 4/5. Combines identity-level repeated collection, distributional metrics, and end-to-end adaptation, with its contribution centered on a complete research benchmark.
Experimental Thoroughness: 4/5. Includes ablations, closed-loop competence, and human preference evidence, but cross-domain validation and reproduction details remain limited.
Writing Quality: 3/5. The main argument is clear, while metric boundaries, some table protocols, and loss specifications need greater precision.
Value: 4/5. Provides a setting in which driving well and driving like a particular person can be tested separately.