Stabilizing Real-World Visual Active Tracking with Action-Smooth Test-Time Adaptation¶

Conference: ECCV 2026
Paper: ECCV Official
Area: Robotics & Embodied AI
Keywords: Visual Active Tracking, Test-Time Adaptation, Reinforcement Learning, Action Smoothness Regularization, Embodied AI

TL;DR¶

Addressing the severe sim-to-real visual domain shifts and distractor confusion in embodied visual active tracking, this paper proposes VATA, the first test-time adaptation framework for VAT that couples critic value maximization for sample filtering with action smoothness regularization for motion continuity, boosting real-world physical drone tracking success rate from 20% to 80%.

Background & Motivation¶

Visual Active Tracking (VAT) requires an embodied agent to actively control its camera pose or robotic platform in dynamic 3D environments to follow a moving target. Serving as a foundational capability for robotic navigation, security surveillance, and aerial videography, VAT operates in an interactive closed-loop perception-action regime, in contrast to passive visual tracking that performs simple bounding box regression over recorded video streams. Modern reinforcement learning (RL)-based VAT trackers integrate perception and low-level control into a unified policy network, enjoying lightweight footprints and real-time execution. However, almost all existing trackers are trained and evaluated in idealized simulators (such as UnrealCV, Webots, or Habitat), suffering severe performance degradation when directly deployed in the physical world.

The core tension behind this sim-to-real collapse stems from two interconnected challenges: severe visual domain shifts and unexpected target distribution shifts. In the real world, complex visual textures, dynamic illumination changes, and camera artifacts confuse the tracker's visual perception. Simultaneously, real-world scenes are replete with visually similar distractors (e.g., pedestrians in identical attire), triggering false tracking and erratic steering. While collecting and annotating on-site training data across diverse environments is economically infeasible, Test-Time Adaptation (TTA) offers a promising unsupervised paradigm by updating models online during inference. However, conventional TTA paradigms (such as TENT, EATA, or TARL) rely on prediction entropy minimization. In real-world VAT, trackers often exhibit strong over-confidence even after completely losing the target, rendering output entropy virtually indistinguishable between success and failure cases. Blindly minimizing entropy thus absorbs noisy samples and corrupts tracking parameters, while lacking physical motion priors to suppress wild steering jerks toward distractors.

This paper tackles these challenges by rethinking the optimization signal and physical kinematics of embodied active tracking. The authors observe that while prediction entropy remains blind to target loss, the critic value in an actor-critic RL architecture drops drastically upon losing the target, serving as an exceptionally discriminative indicator. Furthermore, because physical target motion is inertia-constrained with bounded maximum velocity, optimal control variations must be mathematically bounded. Core idea: develop VATA, the first test-time adaptation framework for visual active tracking, which maximizes discriminative critic values to filter reliable test samples for updating batch normalization layers under visual shift, and imposes an action smoothness regularization derived from target motion continuity to correct erratic controls caused by distractors.

Method¶

Overall Architecture¶

VATA operates entirely online during runtime test deployment. At each interaction step, given an egocentric RGB observation, the actor network predicts camera control actions while the critic network evaluates the expected state or state-action value. The framework evaluates whether the critic value surpasses an empirical threshold to gate a value maximization loss, while simultaneously tracking rotation commands in a sliding action history buffer to detect sudden angular jerks against the running trend. The resulting unsupervised objectives are merged into a unified adaptation loss that updates exclusively the affine parameters of the batch normalization (BN) layers via gradient descent before dispatching smoothed actions to the robot hardware.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Online RGB Observation s_t"] --> B["Actor Policy Network π_θ"]
    B --> C["Predicted Control Action a_t"]
    A & C --> D["Critic Value Estimator Q_θc"]
    D --> E["Critic Value Maximization<br/>Reliable Sample Gating & Loss L_val"]
    C --> F["Sliding Action History Buffer H_t"]
    F & C --> G["Action Smoothness Regularization<br/>Angular Jump Penalty L_smooth"]
    E & G --> H["Combined Adaptation Objective L_total"]
    H --> I["Online Gradient Update<br/>Batch Normalization Affine Parameters Only"]
    I --> J["Execute Smoothed Control on Physical Robot Platform"]

Key Designs¶

1. Critic Value Maximization: Highly Discriminative Adaptation Signal Replacing Entropy Standard perception and RL TTA methods minimize predictive entropy, assuming low entropy corresponds to high prediction confidence. However, empirical statistics reveal that VAT policies remain stubbornly over-confident even when the target has completely vanished from view: the entropy difference between tracking success and tracking failure is negligible (a mere 3.4% on DAT and 3.8% on UnrealCV). In stark contrast, the critic value drops by 66% upon target loss (e.g., plummeting from 9.12 to 3.10 on DAT), providing a reliable discriminative signal. Because critic values on out-of-distribution (OOD) test streams are lower overall than in-distribution baselines, VATA sets a threshold $Q_0$ on the critic value $Q(s_t, a_t)$ to filter trustworthy tracking actions and maximizes their value. For continuous control tasks, the objective directly maximizes the expected Q-value of the actor's output: $$ \mathcal{L}{val}^{cont}(\theta) = - \mathbb{I}}(Q_t) \cdot \mathbb{E{s_t}\left[ Q(s_t, \pi\theta(s_t)) \right] $$ For discrete control spaces, the objective maximizes the state potential: $$ \mathcal{L}{val}^{disc}(\theta) = - \mathbb{I}\left[ Q(s_t) \right] $$ This design updates the perception representations using only high-confidence visual feedback, effectively aligning OOD visual features with training distribution standards without label supervision.}(Q_t) \cdot \mathbb{E}_{s_t

2. Action Smoothness Regularization: Kinematic Continuity Bounding Distractor Jumps In crowded real-world scenes, visually similar distractors frequently trigger sudden perceptual misclassification, causing the camera to jerk abruptly toward the distractor. The authors establish theoretical grounding in Proposition 1: assuming real-world target motion $x_T(t)$ is differentiable with velocity bounded by $v_{max}$ ($\|\dot{x}_T(t)\| \le v_{max}$), and assuming an optimal tracking policy $\pi^*$ is locally Lipschitz continuous with constant $L$, the time derivative of the optimal control command is strictly upper-bounded: $$

|\dot{a}^*(t)| \le L \cdot v_{max} $$ Any control jump exceeding this kinematic limit indicates an irrational command induced by distractor interference. Because distractors primarily induce abrupt orientation deviations, VATA maintains a sliding buffer $H_t = \{r_{t-K}, \dots, r_{t-1}\}$ of rotation actions over the past $K$ steps. Using the moving average $\bar{r}_t = \frac{1}{K}\sum_{i=1}^K r_{t-i}$ to capture historical movement inertia and a scaling factor $\sigma$, the control jump is measured as: $$ d_t = \frac{r_t - \bar{r}t}{\sigma} $$ When the deviation exceeds a tolerance threshold $\delta$, a one-sided ReLU penalty is activated: $$ \mathcal{L}(|d_t| - \delta) $$ This term suppresses irrational rotational jerks, constraining the tracker to maintain orientation along the physical trajectory of the true target.}(\theta) = \text{ReLU

3. Lightweight Batch Normalization Adaptation and Pipeline Extensibility The overall test-time adaptation objective combines value maximization and action smoothness: $$ \mathcal{L}{total} = \lambda_v \mathcal{L} $$ To maintain real-time tracking rates and prevent catastrophic forgetting of pre-trained control representations, adaptation updates only the affine scale and shift parameters of the batch normalization layers, while all backbone weights and control projection heads remain frozen. Furthermore, this action smoothness formulation generalizes beyond RL trackers to imitation-learning (IL) and diffusion-based planners such as TrackVLA. By measuring Euclidean trajectory displacement between newly planned waypoints and the overlapping tail of the previous plan, the same regularization cleanly stabilizes trajectory generation.} + \lambda_s \mathcal{L}_{smooth

Key Experimental Results¶

Main Results¶

VATA was evaluated against the base trackers (EVT / GC-VAT) and prominent TTA baselines (TENT, EATA, COME, TARL) across UnrealCV, DAT, and EVT-Bench benchmarks, followed by physical deployment on a DJI Tello drone.

UnrealCV Benchmark with Distractors (from Table 1 in original paper)

Method	Parking Lot (2D) SR ↑	UrbanCity (4D) SR ↑	ComplexRoom (4D) SR ↑	Average SR ↑	Average AR ↑
EVT (Base Tracker)	0.64	0.86	0.86	0.79	292
TENT	0.48	0.76	0.74	0.66	248
EATA	0.66	0.88	0.86	0.80	316
COME	0.68	0.89	0.88	0.82	321
TARL (Prev. SOTA)	0.70	0.88	0.88	0.82	320
VATA (Ours)	0.74	0.90	0.92	0.85	336

EVT-Bench Multi-Task Suite and Real-World Drone Deployment (from Table 3 & Table 7)

Evaluation Setup / Task	Metric	Base Tracker (EVT)	TARL (SOTA TTA)	VATA (Ours)	Relative Gain
EVT-Bench (Single Target STT)	Success Rate SR (%) ↑	24.3	26.7	30.5	+14.2% vs. TARL
EVT-Bench (Distracted DT)	Success Rate SR (%) ↑	3.4	6.8	11.2	+64.7% vs. TARL
EVT-Bench (Ambiguity AT)	Success Rate SR (%) ↑	17.6	20.6	25.4	+23.3% vs. TARL
EVT-Bench All Tasks Mean	Average SR (%) ↑	15.1	18.0	22.4	+24.4% vs. TARL
Real DJI Tello Drone (10 Episodes)	Tracking Success TSR ↑	0.20 (2/10)	0.30 (3/10)	0.80 (8/10)	4× gain vs. Base

Ablation Study¶

Ablation experiments conducted on UnrealCV systematically verify each component (from Table 4 and Table 6 in original paper).

Config	Parking Lot (2D) SR ↑	UrbanCity (4D) SR ↑	ComplexRoom (4D) SR ↑	Average SR ↑	Average AR ↑	Note
Full Model (Ours)	0.74	0.90	0.92	0.85	336	Both $\mathcal{L}_{val}$ and $\mathcal{L}_{smooth}$ active
w/o $\mathcal{L}_{val}$	0.66	0.87	0.87	0.80	299	SR drops by 5.9%; visual domain adaptation collapses
w/o $\mathcal{L}_{smooth}$	0.70	0.88	0.89	0.82	323	SR drops by 3.5%; tracking vulnerable to distractors
Base Model (EVT)	0.64	0.86	0.86	0.79	292	No test-time adaptation applied

Extensibility tests on Open-TrackVLA further show that adding action smoothness regularization improves SR on the EVT-Bench DT task from 11.6% to 16.9% (+45.7% relative gain), confirming its model-agnostic effectiveness.

Key Findings¶

Critic value exhibits intrinsic fidelity under policy failure: Removing $\mathcal{L}_{val}$ causes the steepest performance collapse (SR drops from 0.85 to 0.80), demonstrating that value functions assess test sample confidence far more faithfully than predictive entropy in closed-loop embodied control.
Distractors represent the primary bottleneck in physical deployment: On the EVT-Bench DT setting, base EVT achieves only 3.4% SR. Adding action smoothness regularization nearly quadruples performance to 11.2%. In real-world physical tests, unregularized trackers invariably veer toward bystanders, while VATA maintains 80% full tracking success.
Robustness across hyperparameter grids: Sweeping $Q_0 \in [5, 10, 20]$ and $\delta \in [0.1, 0.2, 0.3]$ yields less than 3.3% variance in accumulated reward, demonstrating wide operational stability.

Highlights & Insights¶

Revisiting test-time adaptation assumptions in embodied control: Reveals the failure mode of entropy minimization in active perception where agents remain falsely confident during target loss, establishing critic value maximization as a sound foundation for embodied TTA.
Rigorous physical kinematics guiding unsupervised regularization: Exploits target velocity bounds to derive an optimal control variation bound, turning a simple sliding-window ReLU penalty into a mathematically grounded defense against distractor jumps.
Ultra-lightweight real-time physical deployment: Requiring gradient updates solely on BN affine parameters, the system comfortably runs at 30 FPS over Wi-Fi between an RTX 3090 workstation and a DJI Tello drone, offering a plug-and-play sim-to-real recipe.

Limitations & Future Work¶

Vulnerability to prolonged severe occlusions: In outdoor physical testing (Episode 4), VATA lost the target when it remained fully occluded for an extended period. Because purely unsupervised TTA lacks long-term spatial memory or trajectory prediction, re-acquisition remains an open challenge.
Uniaxial rotation smoothing: The smoothness penalty focuses primarily on yaw angular velocity, leaving 3D longitudinal acceleration and braking unconstrained when targets abruptly stop or reverse.
Future Directions: Exploring self-supervised spatial-temporal memory or generative trajectory diffusion models to handle long-term target re-identification during extended occlusions.

vs. TARL / EATA / TENT (General TTA Methods): Prior methods optimize prediction entropy and assume high confidence equals accurate tracking. In active tracking, target loss often preserves low entropy, misleading prior TTA methods into catastrophic drift; VATA replaces entropy with critic values and incorporates physical continuity priors, vastly outperforming TARL (80% vs 30% TSR on real drone).
vs. TrackVLA / FollowAnything (VLM / Pipeline VAT Trackers): Foundation-model-based trackers depend on large backbones (Vicuna-7B, SAM, DINOv2) with prohibitive latency and compute costs; VATA enables lightweight RL trackers to bridge the domain gap with minimal overhead, maintaining 30 FPS closed-loop responsiveness.

Rating¶

Novelty: ⭐⭐⭐⭐☆ (First TTA method tailored for VAT, establishing critic-based filtering and kinematic action smoothness)
Experimental Thoroughness: ⭐⭐⭐⭐⭐ (Evaluated on three major simulation benchmarks, extensive ablations, and real DJI Tello robot tests)
Writing Quality: ⭐⭐⭐⭐⭐ (Clear problem framing, solid mathematical proposition, and transparent empirical analyses)
Value: ⭐⭐⭐⭐⭐ (Provides an immediate, plug-and-play solution for deploying sim-trained embodied agents onto physical robots)