TraversRL: Traversable Pedestrian Pathway Generation With Reinforcement Learning¶
Conference: ECCV 2026
Paper: ECCV 2026 Poster
Area: Autonomous Driving
Keywords: Pedestrian Pathway Extraction, Reinforcement Learning, Aerial Image Analysis, Iterative Graph Generation, GRPO
TL;DR¶
Addressing the fragmentation problem of traditional aerial image segmentation for sidewalk mapping, TraversRL models pedestrian network generation as an iterative sequential decision process from a traveler's perspective, employing a discretized direction-distance action space optimized with GRPO using global graph-level IoU and local stepwise geometric efficiency reward shaping, which more than doubles the connectivity metric TravSim while significantly pruning redundant branches.
Background & Motivation¶
Accurate, topologically connected pedestrian pathway maps comprising sidewalks, crosswalks, and informal cut-throughs form the foundational infrastructure for accessible navigation, micro-mobility analysis, and urban planning. However, ground-truth pedestrian data remains notoriously scarce and labor-intensive to collect. Inferring traversable pedestrian networks directly from aerial imagery presents formidable visual and geometric hurdles: pedestrian pathways are substantially narrower and more curvilinear than vehicular roads, frequently occluded by dense tree canopies, characterized by subtle visual cues or implicit paths (such as unmarked crossings or parking-lot paths), and small local geometric mispredictions easily lead to severe topological disconnects that destroy end-to-end routability.
Existing mapping pipelines rely heavily on a "segmentation-first" paradigm (exemplified by Tile2Net), which predicts dense raster masks and subsequently recovers graph structure via morphological thinning and heuristic post-processing. Such bottom-up approaches frequently yield disconnected, fragmented subgraphs that cannot guarantee network-wide reachability. Conversely, iterative graph exploration frameworks successful in road network extraction (e.g., RoadTracer, VecRoad, NETracer) fail when applied to pedestrian environments because road extraction relies on distinct junctions and consistent road widths, whereas pedestrian paths exhibit high local curvature, dense short links, and weak visual priors at intersections, causing iterative rollouts to suffer rapid localization drift and compounding topological failures.
Formulating pathway generation as a long-horizon decision-making agent introduces a fundamental tension: per-step supervised imitation exhibits an objective mismatch with ultimate network-level traversability, where small step errors compound catastrophically; meanwhile, reinforcement learning can directly optimize non-differentiable graph-level metrics, but pure terminal rewards suffer from severe credit assignment ambiguity and gradient variance across multi-step trajectories. Core idea: model pedestrian pathway extraction from aerial imagery as an iterative decision-making process conditioned on local visual and canvas states, bootstrap a reference policy with noise-injected supervised pretraining, and apply GRPO fine-tuning combining a global buffered IoU terminal reward with an additive local stepwise geometric efficiency advantage shaping term to balance network-wide routability and edge placement precision.
Method¶
Overall Architecture¶
TraversRL adopts a two-stage training paradigm coupled with iterative rollout inference. Conditioned on an intersection-level aerial image crop and a dynamic raster canvas reflecting previously predicted edges, the model iteratively extends the network from an active node by predicting directional displacement and distance, continuing until a stop action is emitted or a per-node expansion budget is exhausted.
The training framework progresses through two stages: first, supervised pretraining (TraversRL-Pre) trains the 4-channel visual backbone to imitate ground-truth node expansions under injected spatial localization noise; second, reinforcement learning fine-tuning initializes from TraversRL-Pre and optimizes the policy via Group Relative Policy Optimization (GRPO). Under a frozen reference policy regularizer, GRPO simultaneously incorporates a global terminal graph reward (buffered IoU) and a local stepwise geometric efficiency reward shaping mechanism, directing policy updates toward topologically coherent and geometrically uncluttered networks.
%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
A["Input: Aerial crop Iu + Canvas crop Cu<br/>Concatenated into 4-channel state Xu"] --> B["4-Channel Vision Backbone<br/>ResNet-18 / ViT-S / Swin-T"]
B --> C["Noise-Injected Supervised Pretraining<br/>Predict 361-dim direction-distance action"]
C --> D["GRPO Group Rollouts & Advantage Computation<br/>KL penalty against frozen reference ฯฮธ0"]
D --> E["Global Buffered IoU Terminal Reward<br/>Evaluates full-graph overlap and topology"]
D --> F["Local Stepwise Geometric Efficiency Shaping<br/>Computes per-step ground-truth area contribution"]
E --> G["Output: Connected and geometrically clean pathway graph"]
F --> G
Key Designs¶
1. 4-Channel State Representation and Discretized Direction-Distance Action Space: Capturing Complex Pedestrian Geometry To ensure the generative policy remains aware of both surrounding aerial visual context and accumulated network topology while preventing duplicate tracing, TraversRL extracts an observation window of \(0.5 \times 0.5\) normalized image coordinates centered at the current node \(u\). The local RGB crop \(I_u\) is concatenated with a single-channel raster canvas \(C_u\) rendering all previously generated edges, forming a 4-channel input tensor \(X_u \in \mathbb{R}^{4 \times H \times W}\). In the action space, pedestrian networks demand both fine short curvilinear adjustments and extended straight crossings. TraversRL discretizes the polar displacement vector \(\Delta = v - u\) into \(N_\theta = 36\) angular bins (\(10^\circ\) resolution) and \(N_r = 10\) radial distance bins spanning \([0, 0.2]\) in normalized image coordinates. Including an explicit \(\text{Stop}\) action, the action space encompasses \(|A| = N_\theta N_r + 1 = 361\) discrete actions. The vision backbone adapts ImageNet-pretrained weights by zero-initializing the added canvas channel, outputting categorical action logits via an MLP classification head.
2. Robust Supervised Pretraining with Localization Perturbation: Mitigating Iterative Rollout Drift Supervised training with strict ground-truth node coordinates introduces exposure bias: during autonomous inference, minor geometric drift places the agent in out-of-distribution states, accelerating compounding tracking errors. To build robustness, TraversRL injects continuous uniform spatial perturbation into the expansion origin during pretraining: $$ \tilde{u} = u + \epsilon, \quad \epsilon \sim \mathcal{U}([-\eta, \eta]^2) $$ with normalized noise scale \(\eta = 0.02\). The observation crop is extracted at \(\tilde{u}\) while the target action class is computed from the adjusted displacement \(\Delta = v - \tilde{u}\). Optimizing cross-entropy loss against perturbed states forces the network to learn corrective steering behaviors, effectively curtailing rollout divergence during autoregressive inference.
3. GRPO Fine-Tuning with Global Graph-Level Reward: Aligning with Non-Differentiable Network Overlap While supervised pretraining establishes local edge continuation competence, end-to-end evaluation relies on non-differentiable graph metrics that depend on decoding complete trajectories. TraversRL leverages Group Relative Policy Optimization (GRPO) to optimize graph-level rewards without maintaining an explicit critic network. For each training intersection, the active policy \(\pi_\theta\) generates a group of \(K = 4\) independent rollouts \(\{\tau_i\}_{i=1}^K\), decoding each into a predicted edge set \(\hat{E}_i\). A terminal reward \(R_i = \text{IoU}_{\text{buf}}(E, \hat{E}_i)\) measures the buffered intersection-over-union (using a 1-meter physical buffer radius, \(r = 0.01\)) against ground-truth edges \(E\). Standardized group-relative advantages are derived as: $$ A_i = \frac{R_i - \bar{R}}{\mathrm{std}({R_j}_{j=1}^K) + \epsilon} $$ By backpropagating advantages through action log-probabilities alongside a per-step KL divergence penalty against frozen reference policy \(\pi_{\theta_0}\), the model systematically upweights trajectories that achieve comprehensive network-wide coverage.
4. Local Stepwise Geometric Efficiency Advantage Shaping: Fine-Grained Credit Assignment Suppresses Spurious Branches A uniform terminal advantage \(A_i\) applies identically across all steps in trajectory \(\tau_i\), creating coarse credit assignment where locally erroneous or redundant branches are reinforced simply because the overall network scored well. To provide granular step-level feedback, TraversRL formulates a stepwise geometric efficiency reward. Let \(P_{t-1}\) and \(P_t\) represent the buffered predicted edge unions before and after step \(t\), and \(G\) denote the ground-truth buffered region. The newly generated buffer area is \(\Delta A_t = \text{area}(P_t) - \text{area}(P_{t-1})\), and its overlapping subset with ground truth is \(\Delta I_t = \text{area}(P_t \cap G) - \text{area}(P_{t-1} \cap G)\). The stepwise reward is: $$ r_t = \frac{\Delta I_t}{\Delta A_t + \epsilon} $$ For non-edge actions (such as \(\text{Stop}\)), \(r_t = 0\). When a newly placed edge aligns with true paths, \(r_t \approx 1\); when it produces spurious off-target geometry, \(r_t \approx 0\). Within each trajectory, \(r_{i,t}\) is standardized to \(\tilde{r}_{i,t}\) and additively combined with global advantage \(A_i\): $$ \hat{A}{i,t} = A_i + \lambda \tilde{r} $$ With shaping coefficient \(\lambda = 0.1\) and reward clipping to \([-3, 3]\), this additive shaping preserves stable global optimization while penalizing unaligned exploration, effectively eliminating spurious spurs and redundant loops.
Loss & Training¶
The training protocol comprises: 1. Supervised Pretraining: Optimized with AdamW (learning rate \(1 \times 10^{-4}\), weight decay \(1 \times 10^{-4}\), gradient clipping 1.0) for 50 epochs with gradient accumulation across 16 node expansions. For ResNet-18 backbones, BatchNorm running statistics are frozen. 2. RL Fine-Tuning: Initialized from TraversRL-Pre and trained for 15 epochs using AdamW (learning rate \(3 \times 10^{-5}\), weight decay \(1 \times 10^{-4}\)) with gradient accumulation over 8 graphs. The objective optimizes shaped advantages under a step-level full-distribution KL regularizer (\(\beta = 0.05\)): $$ \mathcal{L}{\text{local}}(\theta) = -\frac{1}{K}\sum}^K \frac{1}{T_i}\sum_{t=1}^{T_i}\hat{A{i,t}\log\pi\theta(a_{i,t}\mid s_{i,t}) + \beta \cdot \frac{1}{K}\sum_{i=1}^K \frac{1}{T_i}\sum_{t=1}^{T_i} D_{\mathrm{KL}}!\left(\pi_\theta(\cdot\mid s_{i,t})\,\Vert\,\pi_{\theta_0}(\cdot\mid s_{i,t})\right) $$ During inference, rollouts utilize an active node stack, Top-\(k\) geometric validity backoff (\(k=1\) default greedy), multi-start restarts (\(R\) passes retaining highest logit margin), and maximum expansion budgets.
Key Experimental Results¶
Main Results¶
TraversRL was evaluated on the in-domain WashingtonInter benchmark (13 Washington state cities) and two zero-shot transfer test sets from PathwayBench (Seattle and Washington, D.C.). Metrics comprise buffered geometry IoU (strict 1-meter radius) and traversability similarity (TravSim), measuring Jaccard agreement of boundary-to-boundary routability across local tiles (reported as mean \(\pm\) standard deviation over 5 random seeds).
| Dataset | Model / Backbone | Stage | Buffered IoU | TravSim | Relative Performance vs Baseline |
|---|---|---|---|---|---|
| WashingtonInter (In-domain) | Tile2Net (SOTA segmentation) | โ | \(0.264 \pm 0.046\) | \(0.114 \pm 0.017\) | Severe graph fragmentation |
| VecRoad (Road tracing baseline) | โ | \(< 0.001\) | \(< 0.001\) | Failed on subtle pedestrian junctions | |
| NETracer (Tubular tracing) | โ | \(0.098\) | \(0.005\) | Premature termination | |
| TraversRL (ResNet-18) | Pre | \(0.510 \pm 0.003\) | \(0.586 \pm 0.005\) | Pretraining substantially outperforms SOTA | |
| TraversRL (ResNet-18) | Global | \(0.539 \pm 0.001\) | \(0.591 \pm 0.008\) | Global RL consistently improves | |
| TraversRL (ResNet-18) | Local | \(0.540 \pm 0.002\) | \(0.605 \pm 0.004\) | Stepwise shaping boosts connectivity | |
| TraversRL (Swin-T) | Pre | \(0.531 \pm 0.004\) | \(0.605 \pm 0.006\) | Strong hierarchical representations | |
| TraversRL (Swin-T) | Global | \(0.559 \pm 0.002\) | \(0.607 \pm 0.004\) | Marked IoU improvement | |
| TraversRL (Swin-T) | Local | \(0.584 \pm 0.003\) | \(0.611 \pm 0.008\) | Best overall (IoU +121%, TravSim +436% vs Tile2Net) | |
| Seattle (PB) (Zero-shot) | Tile2Net | โ | \(0.169 \pm 0.021\) | \(0.175 \pm 0.037\) | Drops under cross-resolution shift |
| TraversRL (Swin-T) | Pre | \(0.205 \pm 0.004\) | \(0.401 \pm 0.004\) | Zero-shot connectivity doubles | |
| TraversRL (Swin-T) | Global | \(0.218 \pm 0.001\) | \(0.416 \pm 0.004\) | Solid cross-city generalization | |
| TraversRL (Swin-T) | Local | \(0.230 \pm 0.005\) | \(0.418 \pm 0.007\) | Top zero-shot transfer score | |
| D.C. (PB) (Zero-shot) | Tile2Net | โ | \(0.226 \pm 0.018\) | \(0.317 \pm 0.023\) | Baseline segmentation |
| TraversRL (Swin-T) | Pre | \(0.231 \pm 0.002\) | \(0.535 \pm 0.004\) | Preserves robust boundary connectivity | |
| TraversRL (Swin-T) | Local | \(0.252 \pm 0.003\) | \(0.535 \pm 0.003\) | Highest geometric precision |
Ablation Study¶
1. RL Reward Formulation and Graph Cleanliness vs. Connectivity (Averaged Across Backbones and Seeds) Average node degree (AvgDeg, lower indicates sparser/cleaner graphs) and TravSim (higher indicates closer match to ground-truth boundary traversability) illustrate structural evolution under RL:
| Dataset | Training Stage | Average Degree AvgDeg โ | Relative Change vs Pre | TravSim โ | Relative Change vs Pre |
|---|---|---|---|---|---|
| WashingtonInter | TraversRL-Pre | 0.429 | baseline | 0.595 | baseline |
| TraversRL-Global | 0.416 | -3.0% | 0.599 | +0.6% | |
| TraversRL-Local | 0.407 | -5.1% | 0.607 | +2.0% | |
| Seattle (PB) | TraversRL-Pre | 0.574 | baseline | 0.389 | baseline |
| TraversRL-Global | 0.541 | -5.7% | 0.401 | +3.1% | |
| TraversRL-Local | 0.519 | -9.6% | 0.411 | +5.7% | |
| D.C. (PB) | TraversRL-Pre | 0.553 | baseline | 0.513 | baseline |
| TraversRL-Global | 0.528 | -4.5% | 0.516 | +0.6% | |
| TraversRL-Local | 0.520 | -6.0% | 0.521 | +1.6% |
2. Action Space Discretization Granularity (ResNet-18, TraversRL-Local) Fixing distance bins at 10 and varying the number of angular directional bins:
| Action Space Size \(\|A\|\) | Angular Bins | WashingtonInter (IoU / TravSim) | Seattle (PB) (IoU / TravSim) | D.C. (PB) (IoU / TravSim) | Analysis |
|---|---|---|---|---|---|
| \(\|A\| = 181\) | 18 bins (\(20^\circ\)) | 0.448 / 0.581 | 0.174 / 0.385 | 0.195 / 0.503 | Coarse steering induces severe drift and missing links |
| \(\|A\| = 361\) (Default) | 36 bins (\(10^\circ\)) | 0.540 / 0.605 | 0.199 / 0.403 | 0.227 / 0.513 | Optimal tradeoff between precision and generalization |
| \(\|A\| = 721\) | 72 bins (\(5^\circ\)) | 0.569 / 0.618 | 0.198 / 0.383 | 0.237 / 0.520 | Slight in-domain gain but sensitive to cross-city scale |
Key Findings¶
- RL Fine-Tuning Gains Arise from Recovering Missing Ground-Truth Paths: Decomposing boundary-pair relations on WashingtonInter reveals that TraversRL-Pre predicts \(\approx 1.86\) relations per tile, with true positives \(\text{TP} \approx 1.55\), false positives \(\text{FP} \approx 0.31\), and false negatives \(\text{FN} \approx 1.09\) (missed connectivity dominates false alarms by \(\sim 3.5\times\)). TraversRL-Local increases \(\text{TP}\) to \(1.60\) (+3.5%) and reduces \(\text{FN}\) to \(1.04\) (-5.0%). Across scenes where TravSim improves (59% of intersections), 92% exhibit increased TP and 91% show decreased FN.
- Stricter Overlap Buffers Reveal Pronounced Geometric Precision Gains: In the buffer-radius ablation on WashingtonInter, TraversRL-Global's relative gain over Pre scales from +2.1% at 4m buffer, to +3.1% at 2m, and reaches +5.2% under the strict 1m buffer. For TraversRL-Local, the relative improvements surge from +2.5% (4m) to +4.0% (2m) and +6.8% (1m), confirming that local reward shaping enforces tight angular and spatial alignment.
- Road Extraction Pipelines Completely Break Down on Pedestrian Geometry: VecRoad achieved \(<0.001\) IoU due to relying on pronounced junction heatmaps that do not manifest in sidewalks; NETracer achieved only 0.098 IoU and 0.005 TravSim as its tube-continuity thresholds prematurely terminate on narrow or tree-covered footpaths.
Highlights & Insights¶
- Reinforcement Learning Translates Seamlessly to Long-Horizon Aerial Graph Vectorization: Framing pathway synthesis as a verifiable sequential decision problem optimized via GRPO bypasses the differentiability limitations of polygon rendering, outperforming raster segmentation by over \(40\%\) in IoU and \(200\%\) in TravSim.
- Additive Local Advantage Shaping Solves Credit Assignment Without Drift: Normalizing stepwise rewards purely across rollouts causes catastrophic policy collapse due to variable rollout lengths; additively combining within-rollout standardized step rewards \(\tilde{r}_{i,t}\) with global advantage \(A_i\) preserves convergence stability while penalizing spurious off-target spurs.
- Structural Self-Pruning Emerges Naturally: RL fine-tuning actively reduces average node degree by 5% to 9.6% while consistently increasing end-to-end traversability similarity, effectively eliminating redundant branches and noisy cyclic artifacts generated by supervised imitation.
Limitations & Future Work¶
- Fixed Normalized Window Induces Scale Vulnerability: The model operates on a fixed \(0.5 \times 0.5\) crop in normalized image coordinates, making physical resolution dependent on zoom level and ground sampling distance (e.g., zoom 20 in WashingtonInter vs. zoom 19 in PathwayBench). Incorporating explicit physical metric scale embeddings would improve zero-shot transfer.
- Prerequisite of Ground-Truth Start Seeds for Evaluation: Rollouts currently initialize from known ground-truth vertices. Practical zero-shot deployment requires an automated seed proposal module (such as a lightweight segmentation prior or road-sidewalk intersection heuristic).
- Inability to Bridge Physically Disconnected Subgraphs with a Single Seed: A single rollout from one start point can only trace reachable connected components. Complete intersection coverage in complex multi-island networks necessitates multi-seed proposals and unexplored spatial masking.
Related Work & Insights¶
- vs Tile2Net: Tile2Net extracts pedestrian infrastructure via pixel-level semantic segmentation and heuristic vector thinning, inevitably breaking continuity under canopy occlusion or unmarked crossings; TraversRL actively bridges visual gaps via sequential decision rollouts, achieving superior topological completeness.
- vs RoadTracer & VecRoad: Vehicular road graph generators rely on fixed tracing step sizes and distinct junction cues; TraversRL adapts to pedestrian realities by introducing polar direction-distance discretization and local geometric overlap shaping tailored to high-density sidewalk topology.
Rating¶
- Novelty: โญโญโญโญโ [Novel synthesis of GRPO reinforcement learning and local geometric efficiency shaping for aerial pedestrian vector mapping]
- Experimental Thoroughness: โญโญโญโญโญ [Evaluated across 3 backbones, 3 datasets, rigorous 1m buffer and TravSim metrics, supported by comprehensive ablations and error decomposition]
- Writing Quality: โญโญโญโญโญ [Clear mathematical formulations, coherent structure, transparent discussion of limitations and failure modes]
- Value: โญโญโญโญโญ [Provides a practical and scalable solution for generating high-fidelity pedestrian routing networks critical for urban accessibility]