Unified Prediction and Planning via Conflict-Aware Disjoint Parameter Training

Conference: ECCV 2026
Paper: ECCV 2026 Poster
Code: https://dpt2026.github.io/
Area: Robotics & Embodied AI
Keywords: motion prediction, motion planning, model merging, skill conflict, robot navigation

TL;DR

To tackle severe representational skill conflicts in compact shared encoders for robot navigation, this paper introduces Disjoint Parameter Training (DPT) and Sparse Merging, achieving safe ego planning and accurate neighbor prediction without any inference latency overhead.

Background & Motivation

In dense and interactive crowd navigation scenarios, mobile service robots must simultaneously forecast the future trajectories of surrounding pedestrians and plan collision-free, kinematically viable motion for themselves. Conventional navigation systems decouple this pipeline into independent, sequential prediction and planning stages. However, this modular pipeline incurs prohibitive cumulative inference latency on edge platforms and fundamentally cuts off closed-loop mutual feedback between interactive agents. Consequently, modern paradigms are rapidly transitioning toward unified end-to-end architectures that perform both prediction and planning jointly over a single shared encoder backbone.

Nonetheless, deploying unified models on resource-constrained embedded edge devices imposes a strict limit on model capacity. Unlike massive foundation models, compact encoders suffer from an inherent representational tension between the two objectives. Motion prediction requires the encoder representation to be highly sensitive to diverse, multi-agent behavioral cues and subtle pedestrian intentions, whereas motion planning requires ego-centric, risk-averse features dedicated to feasibility and collision avoidance. When both decoders backpropagate conflicting gradients into the same compact backbone, parameter updates interfere destructively. Empirically, when expanding task-specific parameter utilization to 50%, coordinate-level overlap within the shared encoder surges to 39.25%, precipitating a severe "Skill Conflict" that degrades joint performance and leads to premature performance saturation.

Existing model merging techniques, such as Task Arithmetic and Ties Merging, were tailored for large language models where immense parameter spaces naturally accommodate disjoint skills. When applied directly to compact models, these methods fail due to dense local feature coupling and gradient conflicts. The paper's angle of attack is that rather than reconciling conflicting task directions post-hoc, one should actively decouple the parameter allocation space during task fine-tuning. Core idea: Disjoint Parameter Training (DPT) separates the parameter learning regions of prediction and planning during fine-tuning via dynamic binary mask allocation, consolidates them in localized regions, and sparsely merges only the top-magnitude task vectors to eliminate adjacent feature interference.

Method

Overall Architecture

The system constructs a compact unified model upon an extended Social-Transmotion backbone equipped with dual task decoders: a prediction decoder for neighbor agents and a planning decoder for the ego robot. Given multimodal history observations across \(N\) agents (agent 0 as the ego, agents \(1:N-1\) as neighbors) including historical trajectories \(X_{\text{traj}}\), human skeletal poses \(X_{\text{pose}}\), and bounding boxes \(X_{\text{box}}\), the unified network outputs future ego planned trajectories \(Y_{\text{plan}}\) and neighboring agents' forecasted trajectories \(Y_{\text{pred}}\).

The training and deployment pipeline proceeds through two primary stages. First, in Disjoint Parameter Training (DPT), task-specific material models (\(f_{\text{plan}}\) and \(f_{\text{pred}}\)) are fine-tuned alternately starting from a shared pretrained base \(\Theta_0\). Trainable coordinates are dynamically and mutually exclusively claimed based on absolute gradient magnitudes, and subsequently frozen in the second half of training. Second, in Sparse Merging, only the top salient coordinates (e.g., Top 1%~2%) of each task vector are infused into the base model to eliminate spatial interference. Finally, an optional Joint Reasoning (JR) stage optimizes the remaining unassigned neutral parameters using a joint multi-agent loss.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Multimodal History Inputs<br/>(Trajectory, Pose, Box)"] --> B["Pretrained Shared Backbone"]
    B --> C["Disjoint Parameter Training<br/>(Gradient-Guided Mutually Exclusive Masks)"]
    C --> D["Task Material Models<br/>(f_plan & f_pred with Disjoint Support)"]
    D --> E["Sparse Merging<br/>(Retaining Top-K% Salient Task Coordinates)"]
    E --> F["Joint Reasoning Finetuning<br/>(Optimizing Unassigned Neutral Weights)"]
    F --> G["Unified Output<br/>(Safe Ego Trajectory + Accurate Neighbor Predictions)"]

Key Designs

1. Disjoint Parameter Training: Active Parameter Subspace Decoupling Skill Conflict arises because prediction and planning backpropagate conflicting updates to overlapping coordinates in the shared encoder. DPT replaces unconstrained end-to-end multi-task training with an alternating, coordinate-exclusive allocation protocol across total epochs \(E\). During the first phase (\(e \le E/2\)), each task alternately computes its task loss and gradients across all parameters. The algorithm identifies the set of unassigned coordinates currently unclaimed by either task: $$ U = { i \mid M_{\text{plan}}[i] = 0 \land M_{\text{pred}}[i] = 0 } $$ Within this unassigned set \(U\), it identifies the top-\(K\) indices \(I_{\text{top-}K}\) possessing the highest absolute gradient values \(|\nabla_\Theta \mathcal{L}_t|\), permanently registers them into the task's binary mask \(M_t[j] \leftarrow 1\), and performs gradient descent strictly over this masked footprint: $$ f_t \leftarrow f_t - \eta \nabla_\Theta \mathcal{L}_t \odot M_t $$ During the second half (\(e > E/2\)), mask expansion halts, and the model refines its skills exclusively within the fixed, disjoint parameter subspace. This guarantees that the resulting material models \(f_{\text{plan}}\) and \(f_{\text{pred}}\) possess strictly non-overlapping active parameter supports.

2. Sparse Merging: Suppressing Adjacent Representational Interference Although DPT eliminates direct coordinate overlap, dense activation of disjoint weights still causes local feature disruption due to non-linear neural couplings in compact architectures. Naively merging all DPT-updated parameters degrades overall accuracy. To overcome this, the authors introduce Sparse Merging, which extracts only the top-\(K\%\) coordinates by absolute magnitude from each task vector \(\tau_t = \Theta_t - \Theta_0\) via a percentile mask \(M_t^{(K\%)}\): $$ \Theta_{\text{merged}} = \Theta_0 + M_{\text{pred}}^{(K\%)} \odot \tau_{\text{pred}} + M_{\text{plan}}^{(K\%)} \odot \tau_{\text{plan}} $$ By isolating only the highest-signal, skill-critical coordinates (typically \(K=1\% \sim 2\%\)), Sparse Merging suppresses lateral parameter cross-talk and preserves maximum representational purity for both tasks.

3. Unassigned Parameter Joint Reasoning: Preserving Multi-Agent Interactive Synergy While spatial isolation resolves gradient clashes, completely segregating task parameters could compromise the inherent benefit of unified models—cross-task interactive reasoning. To strike an optimal balance, the framework retains unassigned neutral parameters (coordinates where \(M_{\text{plan}}[i] = 0\) and \(M_{\text{pred}}[i] = 0\)) after Sparse Merging. These parameters undergo a subsequent lightweight fine-tuning stage under a joint game-theoretic loss, enabling the shared encoder to capture interaction dependencies without perturbing the specialized core task skills.

Loss & Training

During initial pretraining, the backbone is optimized using a game-theoretic multi-agent loss. During DPT, the planning objective penalizes displacement errors, collision risks, and target deviations, while the prediction objective minimizes ADE/FDE across surrounding pedestrians. Post-merging, the joint reasoning stage freezes the merged core weights and optimizes the remaining inactive weights end-to-end.

Key Experimental Results

Main Results

On the challenging real-world JRDB crowded robotics navigation benchmark, DPT with Sparse Merging was rigorously benchmarked against state-of-the-art unified architectures (DIPP, DTPP), ensemble baselines, and representative model-merging techniques.

Method	Planning ADE ↓	Collision Rate ↓	Planning FDE ↓	Planning Miss Rate ↓	Prediction ADE ↓	Prediction FDE ↓
DIPP [Huang et al., 2024]	0.8048	0.0167	1.3542	0.6940	0.8834	1.6406
DTPP [Huang et al., ICRA 2024]	0.6232	0.0145	1.1338	0.6931	0.7686	1.4253
Ensemble (Dual-Model Output Average)	0.5807	0.0146	0.9000	0.4786	1.1313	1.5258
Task Arithmetic [Ilharco et al., 2022]	0.9387	0.0189	1.1798	0.8887	0.9904	1.3870
Ties Merging [Yadav et al., NeurIPS 2023]	0.8759	0.0167	1.1503	0.9019	0.9463	1.3412
Localize-and-Stitch [He et al., 2024]	0.8838	0.0176	1.1393	0.9487	0.8104	1.2670
T-Switch [Qi et al., CVPR 2025]	0.8047	0.0199	1.1009	0.8784	0.8091	1.2112
DPT + SM (Ours)	0.4044	0.0091	0.7458	0.3706	0.5952	1.0352
DPT + SM + JR (Ours Full)	0.3640	0.0074	0.7105	0.3589	0.5097	0.9396

Ablation Study

The ablation investigates the sensitivity of mask ratio \(K\) in Sparse Merging under a 50:50 DPT parameter allocation on JRDB:

Config (Mask Ratio \(K\))	Planning ADE ↓	Collision Rate ↓	Planning FDE ↓	Prediction ADE ↓	Prediction FDE ↓	Note
\(K = 1\%\)	0.4044	0.0091	0.7458	0.5952	1.0352	Highly localized, robust task preservation
\(K = 2\%\)	0.4041	0.0096	0.7320	0.5753	1.0174	Optimal overall balance point
\(K = 5\%\)	0.4156	0.0100	0.7306	0.5871	1.0322	Minor degradation begins
\(K = 10\%\)	0.4193	0.0100	0.7346	0.5917	1.0442	Gradual onset of adjacent interference
\(K = 20\%\)	0.4213	0.0101	0.7389	0.5920	1.0507	Progressive interference accumulation
\(K = 40\%\)	0.4229	0.0101	0.7410	0.5895	1.0505	Dense overlap constrains ceiling

Furthermore, integrating DPT into existing merging methods yields massive performance recoveries on JRDB: - Task Arithmetic + DPT improves planning ADE from 0.9387 to 0.5865 and prediction ADE from 0.9904 to 0.5849. - Ties Merging + DPT enhances planning ADE from 0.8759 to 0.4193 and drops collision rate from 0.0167 to 0.0100.

Key Findings

• "Less is More" in Compact Merging: Performance peaks at an ultra-sparse mask ratio (\(K=2\%\)) and degrades consistently as density increases toward 40%. Dense parameter inclusions introduce detrimental lateral cross-talk in small networks.
• Constrained Learning Space Forces Information Density: In parameter budget sweeps, allocating smaller learnable subspaces during DPT (e.g., 10% vs. 90%) actually improves individual task performance under 1% sparse merging, forcing the model to concentrate core skills into compact, extractable coordinates.
• Interference is Predominantly Parameter-Level: Output-level ensembling substantially outperforms standard post-hoc merging baselines, proving that the degradation in conventional unified models stems from weight conflict rather than task incompatibility.

Highlights & Insights

• Quantification of Edge Skill Conflict: Explicitly formulates and measures parameter overlap dynamics in compact shared encoders, revealing that unconstrained fine-tuning drives coordinate-wise conflict up to 39.25%.
• Proactive Gradient-Guided Subspace Allocation: Rather than treating model merging as an afterthought, DPT integrates spatial isolation directly into the training loop, establishing a new paradigm for multi-task model synthesis.
• Lightweight Deployment Synergy: Reconciles the conflicting demands of multi-task end-to-end reasoning and strict edge latency constraints by producing a single unified model with zero auxiliary runtime overhead.

Limitations & Future Work

• Author-Acknowledged Limitations: Alternating gradient checks and multi-stage fine-tuning add procedural complexity compared to straightforward joint backpropagation.
• Scope & Assumptions: Primary benchmarks focus on low-speed crowd navigation (2.5 fps); validation on high-speed dynamic traffic environments (e.g., CARLA closed-loop autonomous driving) warrants broader continuous testing.
• Future Directions: Exploring layer-adaptive parameter budget discovery instead of uniform manual ratios, and extending DPT principles to multimodal vision-language-action (VLA) embodied robotic foundations.

Related Work & Insights

• vs DIPP [Huang et al., 2024] & DTPP [Huang et al., 2024]: DIPP and DTPP rely on joint gradient updates across a shared backbone, succumbing to internal skill competition; DPT decouples parameter updates at the coordinate level, reducing planning ADE by over 50%.
• vs Task Arithmetic [Ilharco et al., 2022] & Ties Merging [Yadav et al., 2023]: These LLM-centric merging methods deteriorate when applied to compact networks due to pervasive weight overlap; DPT acts as an upstream pre-merging regularizer that rescues their performance.
• vs Localize-and-Stitch [He et al., 2024]: While Localize-and-Stitch learns a post-hoc selection mask, DPT intervenes directly during the optimization trajectory, preventing representational conflict before it manifests.

Rating

• Novelty: ⭐⭐⭐⭐⭐ Pioneering identification and resolution of parameter-level skill conflicts in compact unified robotic models via pre-merging disjoint training.
• Experimental Thoroughness: ⭐⭐⭐⭐⭐ Rigorously tested across JRDB and JTA datasets with comprehensive ablation of mask ratios, parameter allocations, and merging backbones.
• Writing Quality: ⭐⭐⭐⭐⭐ Cohesive narrative progression from empirical conflict diagnosis to mathematical formulation and validation.
• Value: ⭐⭐⭐⭐⭐ Provides an impactful, highly practical framework for deploying safe and performant multi-task robotic systems on edge compute.