CoMaTrack: Competitive Multi-Agent Game-Theoretic Tracking with Vision-Language-Action Models

Conference: ECCV 2026
Paper: Official paper page · PDF
Authors: Li Gao, Liu Liu, Mingyang Lv, Yang Cai
Area: Robotics / Embodied Visual Tracking
Keywords: Competitive training, asymmetric rewards, GRPO, continuous waypoints, multi-view memory

TL;DR

CoMaTrack trains two VLA robots to compete for following the same language-specified target, using asymmetric rewards and SFT-anchored GRPO as a dynamic curriculum; its 3B tracker reaches 92.1%, 74.2%, and 57.5% success on EVT-Bench STT, DT, and AT, with same-model STT success increasing from 89.5% under single-agent RL to 92.1%.

Background & Motivation

Embodied visual tracking is not simply drawing a bounding box in a video. A robot must identify someone from a description, choose its own motion, and maintain visibility and a safe following distance despite occlusion, target movement, and other people. Methods such as TrackVLA supervise a vision-language-action model with expert trajectories, bringing target understanding and waypoint prediction into one network. However, the situations represented in those demonstrations largely determine what the policy learns; rare recovery behaviors and pursuit paths occupied by another robot are difficult to cover thoroughly.

Single-agent RL introduces closed-loop feedback, but it does not automatically remove this distributional bottleneck. If target behaviors and disturbance generators remain fixed, the agent may still adapt to a limited repertoire of pursuit situations. What is missing is not more undirected motion noise, but an interaction partner that continues to exert useful pressure as the tracker improves. CoMaTrack therefore delegates challenge generation to another learning robot, making favorable following positions a resource that both robots seek.

Adversarial games are not new to visual tracking: the authors explicitly build on AD-VAT's asymmetric dueling principle. The new emphasis is the combination of language conditioning, modern VLA representations, continuous waypoint outputs, and online GRPO. Core idea: let an opponent create purposeful route competition by approaching the same target more closely, then train the tracker to handle that evolving interference under safety constraints instead of relying only on a larger static demonstration set.

Method

Overall Architecture

The input consists of a natural-language target description, current front/rear/left/right RGB images, and a sliding window of historical front-view images. The output is a sequence of 5 continuous waypoints, each specifying planar displacement and heading in the robot's coordinate frame. Multi-task supervised learning first produces a VLA capable of identifying targets and planning motion; tracker and opponent policies are then initialized from it and jointly collect trajectories in Habitat while receiving separate policy updates.

The training pipeline connects three components: Multi-View Temporal VLA, Asymmetric Competitive Rewards, and Dual-Policy GRPO. The deployed task policy remains a single tracker; multi-agent interaction primarily shapes its training. The new benchmark separately evaluates whether it can maintain tracking when an opponent is present.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    Input["Target description<br/>Four current views and front-view history"] --> VLA["Multi-View Temporal VLA<br/>Multi-task supervision and waypoint prediction"]
    VLA --> Rollout["Tracker and opponent<br/>Joint environment rollouts"]
    Rollout --> Reward["Asymmetric Competitive Rewards<br/>Safe following and close-range contention"]
    Reward --> GRPO["Dual-Policy GRPO<br/>Separate returns and frozen SFT reference"]
    GRPO --> Rollout
    GRPO --> Output["Deploy the tracker<br/>Predict continuous waypoints"]

Key Designs

1. Multi-View Temporal VLA: preserve current spatial detail while compressing past observations

A tracking failure can arise from distinct information gaps: the target may have just moved from the front to the side, or its identity may become uncertain after a brief occlusion. CoMaTrack therefore combines four current views with historical front-view observations rather than processing only a current image. After Qwen2.5VL-3B extracts visual features, multi-scale grid pooling preserves fine-grained spatial information for recent observations and uses coarse tokens for more distant history. This avoids prematurely discarding current appearance and position cues while preventing older context from consuming an unbounded sequence budget. The cache describes this coarse/fine temporal arrangement, but does not provide a complete reproducible memory-window length or token budget.

Visual tokens enter the language model alongside language embeddings and a navigation task indicator. Recognition tasks use the standard autoregressive text branch; navigation and tracking use a flow-matching action head conditioned on the visual-language representations. Its outputs are continuous motion plans rather than choices from a discrete menu such as left, right, and forward. Each waypoint contains \((x,y,\theta)\), and 5 consecutive waypoints describe the upcoming short motion sequence. The action head builds on general flow matching; inventing that generative mechanism is not the paper's contribution.

Supervised training combines tracking, navigation, and VQA: tracking demonstrations teach how to move, while visual-language tasks preserve the ability to identify targets and understand descriptions. Subsequent RL does not retrain the entire action system. It updates only LLM LoRA adapters, freezing the visual encoder and action head. The same head can therefore produce different waypoints when its conditioning representation changes, with policy adaptation occurring upstream of action generation.

2. Asymmetric Competitive Rewards: create positional contention without rewarding unsafe pursuit by the tracker

Both robots follow the same human target during training. The opponent is neither the tracked person nor a target policy whose sole purpose is to escape. Instead, by occupying closer following positions, it indirectly creates occlusion, path crossings, and physical blocking. Both policies use the same VLA architecture but prefer different distances: the tracker's distance reward is centered at 2.25 m, whereas the opponent's center shifts to 1.25 m. This encourages the opponent to occupy the space nearer the target without rewarding the tracker simply for following as closely as possible.

The single-agent base reward combines distance, facing direction, and tracking persistence. Its distance term is Gaussian, centered at the preferred distance with a width of 0.75 m; departing from the preferred following region reduces this reward. Additional terms reward facing the target and remaining in the safe following zone over consecutive steps. Terminal rewards distinguish successful completion from failures such as target loss or collision. Organizing the readable formula fragments according to the cache's prose gives:

\[ r_{\mathrm{distance}}=w_{\mathrm{distance}}\exp\!\left[-\frac{1}{2}\left(\frac{d-d_{\mathrm{opt}}}{\sigma}\right)^2\right],\qquad d_{\mathrm{opt}}=2.25\,\mathrm{m},\quad\sigma=0.75\,\mathrm{m}. \]

Here \(d\) is the robot-to-target distance and \(w_{\mathrm{distance}}\) is the distance-reward weight. The cached equation has missing characters, so this expression is reconstructed from the explicitly described Gaussian center and parameters, not verified against the PDF typesetting. Other reward weights are not fully provided in the cache; a complete numerical reward function cannot be reconstructed reliably. The multi-agent tracker retains its base reward and adds an opponent-aware safety term penalizing unsafe proximity or collision, while the opponent uses the nearer distance optimum.

This is competitive training, but not necessarily a strictly zero-sum game: the opponent's return is not defined as the negative of the tracker's return, and the paper does not solve for a Nash equilibrium. The operative mechanism changes the states encountered during training, forcing the tracker to navigate around a robot occupying its route without losing the target. It is not an additional abstract game-reasoning module.

3. Dual-Policy GRPO: collect joint experience, optimize separately, and constrain policy drift

When both policies act in the environment, the interference experienced by the tracker changes as the opponent learns. The opponent likewise encounters an increasingly capable tracker. The paper uses synchronized on-policy updates: both policies jointly generate interaction trajectories, then each receives GRPO updates based on its own trajectory returns rather than a shared team reward. Group-relative returns provide a relative preference signal for action selection, while clipping the policy ratio limits individual update magnitudes. The default group size is \(K=10\). The cache does not clearly explain how grouped sampling connects to probability evaluation for continuous flow-matching actions, leaving an important reproduction interface unresolved.

To prevent competition from destroying existing target-understanding and navigation skills, both policies receive a KL constraint against a frozen SFT reference, together with an entropy bonus to preserve exploration. The reference is the supervised policy, not the previous opponent. It limits how quickly each policy departs from its established capabilities. This is particularly relevant to multi-agent non-stationarity: both policies are already changing, and large updates would make the learning conditions change even faster.

The paper reports using the k3 estimator for KL, but the cached GRPO and total-loss equations are severely damaged. Their full symbols and signs cannot be recovered reliably, so this note retains the mechanism, explicit hyperparameters, and ablation evidence rather than inventing an implementation-ready objective. Nor does using KL establish stable convergence: the method section describes stability as comparable to single-agent RL, while the limitations still acknowledge the cost and instability caused by non-stationarity.

A Worked Example

Suppose an instruction identifies someone by their clothing and another robot occupies the space between the tracker and that person. The Multi-View Temporal VLA combines current side views with front-view history to maintain target cues, and the action head predicts 5 waypoints containing displacement and heading. After motion produces new observations, the tracker replans from the updated state instead of executing its initial route to completion without feedback.

During training, the opponent is encouraged toward its 1.25 m reward center and may continue to occupy the direct path behind the target. The tracker instead learns to maneuver around it using collision and facing feedback while preferring a distance of 2.25 m. Their trajectories receive separate returns before Dual-Policy GRPO updates. This is an explanatory walkthrough of the components, not a published individual trajectory or a claim that a particular detour direction was quantitatively verified.

Loss & Training

The supervised objective combines waypoint regression with text cross-entropy. The reliably recoverable relationship is:

\[ \mathcal{L}_{\mathrm{SFT}}=\mathcal{L}_{\mathrm{track}}+\alpha\mathcal{L}_{\mathrm{text}}. \]

The coefficient \(\alpha\) balances the tasks. The method text calls the waypoint loss mean squared error, the illustration labels it L1, and the extracted equation is incomplete. Its actual norm, squaring, and reduction cannot therefore be established from the cache; reproduction requires checking the authors' implementation.

Tracking data include 6913 STT, 6685 DT, and 6524 AT episodes from HM3D and MP3D scenes. Additional sources include ScanQA, LLaVA-Pretrain, SYNTH-PEDES, RefCOCO, Flickr30k, and navigation data from R2R-CE, RxR-CE, ObjectNav, and OVON. SFT runs for 1 epoch on 96 H20 GPUs for approximately 15 hours, using AdamW with learning rate \(10^{-5}\), weight decay \(10^{-3}\), and cosine scheduling.

RL uses 8 L20 GPUs and trains only LLM LoRA adapters, updating after every 20 rollouts. One epoch contains approximately 7000 rollouts and about 1 million environment steps. AdamW again uses learning rate \(10^{-5}\); the clipping coefficient is 0.2 and the entropy coefficient is 0.01. The KL weight increases from 0.02 to 0.10, with an adaptive target KL of 0.01. These are training configurations: a smaller 3B model should not be confused with low training cost.

Key Experimental Results

Main Results

STT, DT, and AT denote Single-Target Tracking, Distracted Tracking, and Ambiguity Tracking. SR is success rate, TR tracking rate, and CR collision rate. All entries below are percentages; higher SR/TR and lower CR are better. The task formulation requires successful following to maintain a safe distance of 1–3 m, keep the target in front of the robot's view, and avoid collisions. The cache does not fully restate the denominators used for TR/CR, so more detailed calculation rules are not invented here.

EVT-Bench task	TrackVLA++ 7B: SR / TR / CR	CoMaTrack 3B: SR / TR / CR	SR change (percentage points)
STT	90.9 / 82.7 / 1.5	92.1 / 90.3 / 0.9	+1.2
DT	74.0 / 73.7 / 3.5	74.2 / 80.5 / 2.1	+0.2
AT	55.9 / 63.8 / 15.1	57.5 / 73.4 / 12.0	+1.6

CoMaTrack-Bench builds on EVT-Bench STT episodes, placing a second robot 0.5 m ahead of the tracker's starting position. It defines static-obstacle, random-interference, and competitive-tracking opponents. The competitive evaluation opponent loads an SFT policy and should not be confused with the continually updated training opponent. Original Table 2 labels the evaluation zero-shot and reports SR/TR/CR of 85.0/82.9/5.5 for CoMaTrack versus 42.4/56.5/23.8 for Uni-NaVid. However, the summary table does not provide complete per-setting results or aggregation weights. The benchmark compares only against Uni-NaVid, which the authors describe as having publicly available weights; this does not establish superiority over every strong baseline.

Ablation Study

Original Table 3 compares training stages within the same system on EVT-Bench STT, providing more direct evidence for the multi-agent contribution than cross-model-size comparisons.

Configuration	SR	TR	CR
SFT only	88.2	85.4	3.1
Single-agent RL	89.5	88.0	2.2
Multi-agent RL	92.1	90.3	0.9

Original Table 4 further examines RL settings; changes are relative to the full model and measured in percentage points.

Configuration	SR	SR change
Full model, default \(K=10\)	92.1	0.0
Group size \(K=5\)	91.1	-1.0
Group size \(K=20\)	92.1	0.0
Without KL constraint	90.7	-1.4
Without entropy bonus	88.4	-3.7
Without KL constraint and SFT reference	87.6	-4.5

Key Findings

• Multi-agent RL adds 2.6 percentage points of STT success over single-agent RL and reduces collision rate from 2.2% to 0.9%. Relative to SFT only, success improves by 3.9 points. The benefits cannot all be attributed simply to introducing RL; the competitive stage adds further gains.
• DT success exceeds TrackVLA++ by only 0.2 points, but tracking rate improves by 6.8 points; AT tracking rate improves by 9.6 points. More persistent tracking reveals benefits that terminal success alone does not fully capture.
• Removing the entropy bonus costs 3.7 points, more than the 1.4-point loss from removing KL alone. Increasing the group size from 10 to 20 provides no further gain, so enlarging the sampling group is not sufficient to address the remaining limitations.
• Section 5.3 reports marginal improvement from static obstacles, slight degradation from random interference, and the best performance from competitive tracking. The conclusion instead summarizes a stepwise improvement across all three. The detailed experimental discussion takes precedence here; Figure 4's numerical coordinates are not readable in the cache and are not fabricated.

Highlights & Insights

• The opponent acts as a task-directed hard-example generator. Rather than injecting arbitrary noise, it seeks favorable positions and forces the tracker to handle occlusions and route conflicts directly related to following behavior.
• Asymmetric distance preferences separate contention from the tracker's safety objective. A transferable idea for shared-space navigation is to generate conflicts through distinct, interpretable resource preferences while preserving the main agent's safety constraints; that transfer is an inference, not an evaluated result here.
• Updating upstream LoRA while freezing the action head illustrates how changes in visual-language conditioning can improve closed-loop behavior. It does not show that the action head is unimportant, since its architecture and unfreezing strategy are not ablated.

Limitations & Future Work

• The authors acknowledge that validation mainly concerns tracking and its competitive variant, without large-scale instruction-navigation or object-navigation experiments. Including those tasks in SFT data does not establish downstream cross-task gains.
• Opponents remain constrained by simulator priors, and multi-agent non-stationarity introduces computational cost and potential instability. More opponent types, independent random seeds, and learning-curve variance would strengthen robustness evidence beyond a single improving trend.
• Strong-baseline coverage on the new benchmark is limited. The 0.2-point DT success lead also lacks confidence intervals and should not be described as statistically significant. Comparisons between 3B and 7B models additionally mix differences in training data, architecture, and optimization.
• Real deployment uses a Unitree GO2 X with four RGB views and streams video to a remote RTX 4090. Demonstrations cover similar distractors, obstacles, and dark constrained spaces, but are mainly qualitative and lack real-world success rates, latency measurements, and network-failure analysis.
• The continuous-action GRPO probability interface, complete reward weights, waypoint loss definition, and parts of the evaluation aggregation are underspecified. Reproduction should resolve these interfaces first; “game-theoretic” should not be mistaken for a formal equilibrium solution or convergence guarantee.

Related Work & Insights

• Compared with AD-VAT / AD-VAT+: asymmetric dueling already exists. CoMaTrack connects that principle to language-conditioned VLAs and continuous waypoint prediction with online GRPO. Its novelty lies in this adaptation and evaluation, not in being the first tracker trained with an opponent.
• Compared with TrackVLA / TrackVLA++: the former emphasizes unified target identification and action planning, while the latter adds spatial reasoning and identity memory. CoMaTrack instead changes the training distribution through interaction. These directions may be complementary, but no combined experiment establishes additive gains.
• Compared with single-agent navigation RL: both use environmental feedback to reduce closed-loop errors, but here the opponent's policy also changes the distribution of future states. The transferable research question is which opponents provide useful learning pressure, not simply whether to add more agents.

Rating

• Novelty: 4/5. Combining language-conditioned VLAs with competitive GRPO is valuable, although the underlying asymmetric dueling idea comes from prior work.
• Experimental Thoroughness: 3/5. Standard benchmarks, training-stage comparisons, and regularization ablations are useful, but competitive baselines, statistical tests, and quantitative real-world evaluation remain limited.
• Writing Quality: 3/5. The overall pipeline is understandable, but reward and probability interfaces are underspecified, and the loss description and random-opponent conclusions are inconsistent.
• Value: 4/5. The work offers a useful dynamic hard-example training paradigm for embodied tracking, with reproduction costs and deployment boundaries that need careful evaluation.