VLA-R1: Enhancing Reasoning in Vision-Language-Action Models¶
Conference: ECCV2026
Paper: ECCV Official Page
Area: Robotics & Embodied AI
Keywords: vision-language-action models, embodied chain-of-thought, reinforcement learning from verifiable rewards, affordance localization, trajectory prediction
TL;DR¶
Built on Qwen2.5-VL-3B, VLA-R1 first learns geometrically grounded embodied reasoning from VLA-CoT-13K and then uses GRPO to optimize region, trajectory, and format rewards, reaching 36.51% IoU and 91.74 average trajectory error in-domain, although its real-robot results require caution about small evaluation samples and inconsistent reported numbers.
Background & Motivation¶
Mapping images and instructions directly to actions does not ensure that a robot has correctly resolved which object to pick, where to contact it, or which container to use. Similar colors, repeated instances, and multiple candidate receptacles turn an apparently simple pick-and-place instruction into a joint problem of reference resolution and geometric constraints. General-purpose vision-language models recognize semantics but may not produce precise interaction boxes or end-effector paths. Supervised fine-tuning of specialized models improves action prediction without necessarily teaching why a particular grasping part is feasible or why a path should avoid an obstacle.
The paper does not introduce a new low-level robot controller; it changes the post-training of a high-level vision-language model. Chain-of-thought (CoT) text alone may teach fluent explanations while leaving coordinates incorrect. Output rewards alone lack a reasoning prior that connects instruction decomposition, object localization, and motion constraints. Trajectory errors are particularly difficult to reduce to endpoint displacement: even when several points are near the target, their order, local directions, and neighboring step lengths can remain inappropriate. A single overlap score or pointwise distance cannot capture all these issues.
The method assigns different roles to data supervision and reward supervision. The former turns existing geometric labels into reasoning examples, while the latter improves the policy according to how closely its final regions and trajectories match annotations, rather than asking another language model whether an explanation sounds convincing. Core Idea: establish a reasoning prior with embodied CoT aligned to geometric annotations, then use verifiable region, trajectory, and output-format rewards to favor more reliable reasoning-and-action outputs.
Method¶
Overall Architecture¶
The input is a scene image and a natural-language manipulation instruction. The model generates explicit reasoning followed by a task-specific 2D affordance interaction box or 2D trajectory. Here, an action is initially an image-space geometric specification, not a sequence of joint angles generated entirely by the model. A downstream control stack parses the output, back-projects it into 3D using depth and camera intrinsics and extrinsics, and applies robot kinematics to obtain executable control commands.
Training proceeds through embodied CoT synthesis, supervised fine-tuning (SFT), and reinforcement learning (RL). Qwen2.5-VL-72B is the offline data teacher, whereas Qwen2.5-VL-3B is the backbone being trained. The teacher generates explanations according to whether the annotation describes an affordance or a trajectory, producing VLA-CoT-13K. SFT learns both reasoning and action segments. Group Relative Policy Optimization (GRPO) then samples multiple candidates for each input, compares them through multi-objective verifiable rewards, and updates the policy. Inference requires neither the teacher nor ground-truth labels, and the geometric reward evaluator is not an online deployment controller.
%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
A["Image, instruction, and geometric labels"] --> B["Embodied CoT Synthesis"]
B --> C["SFT model<br/>Sample candidate outputs"]
C --> D["Multi-Objective Verifiable Rewards"]
A -.->|Training-only verification| D
D --> E["GRPO Policy Optimization"]
E --> F["Deployment: image and instruction<br/>Generate reasoning and geometry"]
F --> G["Depth back-projection and kinematics<br/>Robot execution"]
Key Designs¶
1. Embodied CoT Synthesis: anchor explanations to actual interaction labels
VLA-CoT-13K does not ask the teacher to freely generate robotic common sense independently of action labels. It adds explanations to ShareRobot-style visual and geometric supervision. For affordance samples, the teacher decomposes the instruction into the immediate atomic intention, identifies the target, distractors, and possible contact surfaces, localizes the part that supports the intended interaction, and checks whether occlusion, reachability, and the current object state permit it. For example, picking up a spoon requires identifying a suitable grasping part rather than merely enclosing the entire spoon. The existing region annotation anchors the explanation, limiting the opportunity to mistake linguistic plausibility for geometric correctness.
Trajectory samples instead require explaining how to move from the current position to the target. The teacher converts the instruction into sub-trajectory objectives, locates the gripper and target, considers interactions between the path and the scene, including obstacles, waypoints, and potential collisions, and finally analyzes motion logic, continuity, and kinematic feasibility. The two branches supply reasoning for region selection and motion planning separately; they do not imply that every training sample contains both types of label. The paper describes this task-aware synthesis process but does not fully specify automatic filtering thresholds or an independent quality audit of teacher explanations. Alignment with annotations is a data-generation constraint, not a formal proof that every reasoning statement is correct.
2. Multi-Objective Verifiable Rewards: check regions, path geometry, and parseability separately
The region reward uses generalized intersection over union (GIoU) between predicted and annotated boxes. Ordinary IoU is zero whenever two boxes do not overlap, making it unable to distinguish a near miss from a distant prediction. GIoU additionally considers their smallest enclosing box and therefore assigns different scores to non-overlapping predictions with different degrees of misalignment. It still verifies geometric localization rather than proving every explanation about reachability, physical stability, or object selection. The trajectory reward uses Angle-Length Augmented Fréchet distance (ALAF). Each waypoint carries a position, a local unit motion direction, and a local segment length. Order-preserving path matching considers positional distance, the angle between directions, and a logarithmic penalty on segment-length ratios. A candidate with a correct endpoint but abrupt directional or step-length changes is thus not rewarded solely for reaching near the endpoint.
ALAF retains the discrete Fréchet principle of finding the best worst-case matching cost over all order-preserving correspondences, with separate weights for direction and length penalties. Endpoint directions come from adjacent segments, interior directions blend neighboring segment directions, and the final point uses the preceding segment length. This is not merely an unordered comparison of two point sets. A separate format reward checks for <think>...</think> followed by <output>...</output>, assigning 1 for compliance and 0 otherwise so that the action parser can read the geometric output. Equations (3)–(4) in the local text are visibly corrupted by extraction: Figure 2 expresses the trajectory reward as the complement of normalized distance, whereas the extracted Equation (4) omits that relationship and conflicts with the accompanying higher-is-better description. This note therefore preserves the recoverable geometric mechanism without reconstructing a damaged expression as the authors' exact equation. Reward aggregation weights and cross-task activation details are also insufficiently specified.
3. GRPO Policy Optimization: update generation from relative performance on the same input
After SFT, the old policy generates a group of outputs for the same image and instruction. Each contains a reasoning segment and a geometric action segment and receives rewards under the preceding rules. GRPO compares candidates against the average performance within that input's group, rather than treating raw scores from different tasks as a common difficulty scale. Above-average candidates receive positive updates, while below-average candidates are discouraged. Normalization by the group mean and standard deviation exploits differences among candidates for the same problem without requiring a separately trained state-value network.
Updates are constrained by clipping the new-to-old policy probability ratio and by a KL penalty relative to a frozen reference model. These constraints limit changes that might improve geometric scores while excessively disrupting learned language or formatting behavior. The text mentions process supervision and token-level advantage assignment, but its explicit verifiable rules primarily evaluate final geometry and format. It does not clearly specify independent, reliable process labels for each reasoning statement. The more defensible interpretation is that final output quality reinforces the model while the CoT prior guides joint adaptation of reasoning and action generation; the experiments do not separately establish improved correctness at every intermediate reasoning step.
A Worked Example¶
The affordance example in Figure 5 asks the robot to pick up a blue spoon and put it into a pot. The model must distinguish the target from a red spoon and other distractors before placing the interaction region on a graspable part of the handle. The CoT output, after denormalization, is {"x": 316, "y": 321, "width": 63, "height": 41}, whereas the direct-prediction comparison is {"x": 490, "y": 306, "width": 44, "height": 73}. These coordinates belong to the paper's example, not a grasping template applicable to arbitrary scenes.
Across training and execution, the teacher first uses the original region annotation to explain selecting the handle, and SFT learns the explanation and output structure. GRPO subsequently compares geometric scores and formatting across candidate boxes. At deployment, the model generates a box from only the current image and instruction; the controller must still read depth at that location and solve for robot motion. The example demonstrates differences in target disambiguation and grasp-region selection. The box alone does not establish successful execution of the subsequent full trajectory into the pot.
Loss & Training¶
The first stage performs SFT on VLA-CoT-13K, supervising both <think> and <output> segments so that the model learns task explanations and structured geometric coordinates. The visual backbone and language decoder inherit Qwen2.5-VL-3B's multimodal capabilities. The main changes concern embodied data, post-training rewards, and the action interface rather than a new visual encoder architecture. The second stage applies GRPO as described above, with a frozen reference model limiting policy drift. The teacher is used only for earlier data generation.
The paper does not provide a fully reproducible learning rate, training-step count, group size, reward weights, numerical ALAF weights, or GPU budget. The extracted GRPO equations are also damaged, so this note does not reconstruct the complete loss. The benchmark description says that ShareRobot subsets support CoT generation, training, and evaluation, but does not explicitly list training/test splits or deduplication rules. These are reproducibility gaps, not grounds to assume that every listed image is an independent test sample.
Key Experimental Results¶
Main Results¶
The in-domain benchmark uses ShareRobot. The text lists 6,522 affordance images and 6,870 trajectory images covering 12 robots and 102 tasks, without specifying the exact training/test split. Out-of-domain affordance evaluation uses 1,200 UMD Part Affordance samples covering grasp, cut, pound, and scoop. Out-of-domain trajectory evaluation uses 500 manually corrected VAIT samples. The two tasks are not evaluated on the same test set.
IoU is the percentage overlap between regions, with higher values preferred. Trajectories are evaluated on a [0, 1000) coordinate scale: discrete Fréchet distance (DFD) measures order-preserving shape deviation, Hausdorff distance (HD) measures maximum point-set deviation, and root mean square error (RMSE) measures average error. Avg is their arithmetic mean, with lower values preferred; it is neither millimeter error nor execution success rate. The following selection comes from Table 2. General-purpose models use few-shot prompting, while specialized models have different post-training histories, so this is not an equal-compute ablation.
| Model | In-domain IoU (%) ↑ | In-domain Avg ↓ | Out-of-domain IoU (%) ↑ | Out-of-domain Avg ↓ |
|---|---|---|---|---|
| Qwen2.5-VL-3B-Instruct | 6.15 | 177.09 | 23.96 | 185.76 |
| NORA-3B | 23.48 | 119.79 | 22.44 | 126.77 |
| ManipLVM-R1-3B | 31.00 | 110.87 | 28.00 | 131.99 |
| Embodied-R1-3B | Not reported | 110.75 | Not reported | 126.13 |
| VLA-R1-3B | 36.51 | 91.74 | 33.96 | 93.90 |
Relative to ManipLVM-R1-3B, in-domain IoU increases by 5.51 percentage points, approximately a 17.78% relative improvement. Avg decreases from 110.87 to 91.74, approximately a 17.25% relative reduction. These two percentages highlighted by the paper refer to ManipLVM-R1, not Embodied-R1, whose trajectory Avg is slightly better than that baseline's in the same table.
Real-robot and RoboTwin evaluations report task success rate (SR). Affordance success requires localizing and grasping the correct object, or returning no box when no target exists. Trajectory success requires reaching the goal region and delivering the object. Table 3 uses 10 trials in each of four real-world scenarios and 10 trials each for Piper and UR5 in simulation. Pi0/Pi0.5, OpenVLA, and RT-2-X additionally receive adaptation using 160 collected real-world trajectories and 80 simulated trajectories, whereas the authors state that VLA-R1 is trained only on real-world data. Training conditions therefore differ.
| Real-world protocol and metric | Comparator | Comparator SR (%) | VLA-R1 SR (%) |
|---|---|---|---|
| Table 3: four scenarios, 10 trials each, affordance | Pi0.5 | 62.5 | 62.5 (as reported) |
| Table 3: four scenarios, 10 trials each, trajectory | Pi0.5 | 70.0 | 75.0 |
| Table 4: five scenarios, 30 trials each, affordance | Embodied-R1 | 63.3 | 68.7 |
| Table 4: five scenarios, 30 trials each, trajectory | Embodied-R1 | 68.0 | 71.3 |
An inconsistency in the source must be retained: Table 3 lists VLA-R1 affordance results of 80%, 60%, 70%, and 60%, whose equally weighted mean is 67.5%, but both the Avg column and the narrative report 62.5%. This note preserves the reported value and flags the contradiction rather than silently replacing it with 67.5%. Table 4 adds the color-and-spatial-relation disambiguation scenario S5 and changes the number of trials, so its averages cannot be treated as an upgraded model's results under the same protocol as Table 3.
Ablation Study¶
Table 5 compares embodied CoT and RL under identical hyperparameters, using the same metrics as Table 2. All values below are retained from the original table.
| Config | IoU (%) ↑ | DFD ↓ | HD ↓ | RMSE ↓ | Avg ↓ |
|---|---|---|---|---|---|
| Without embodied CoT or RL | 23.74 | 149.38 | 135.72 | 100.04 | 128.38 |
| Embodied CoT only | 28.37 | 145.51 | 131.26 | 97.03 | 124.60 |
| RL only | 31.85 | 124.90 | 116.35 | 84.72 | 108.66 |
| Embodied CoT + RL | 36.51 | 106.20 | 97.90 | 71.12 | 91.74 |
Key Findings¶
- CoT alone noticeably improves region localization, but trajectory Avg falls only from 128.38 to 124.60. Producing an explanation does not imply an already precise path.
- Adding CoT to RL alone raises IoU by another 4.66 percentage points and reduces Avg by another 16.92. This supports complementarity, but does not establish step-by-step reasoning faithfulness.
- No ablations separately remove GIoU, ALAF's angle term, its length term, or format rewards, so Table 5 cannot identify the most influential reward component.
Highlights & Insights¶
- Supervision is built around the interaction regions and paths that a robot needs, rather than merely adding generic verbal explanations. Data synthesis and post-training optimization therefore target the same kinds of geometry.
- ALAF evaluates trajectory order alongside local geometry, suggesting that embodied rewards should check path structure. Transferability still depends on whether an annotated path represents a reasonable action rather than the only correct action.
Limitations & Future Work¶
- The authors explicitly acknowledge that other platforms, including bimanual robots, have not yet been developed or validated for the method. Generalization across embodiments requires further study.
- Note analysis: real-world evaluations are small and lack confidence intervals, while Table 3 contains an inconsistent average. Advantages of a few percentage points should not be presented as established general reliability.
- Note analysis: missing explicit data splits, full training budgets, and reward-level ablations make it difficult to separate the causal contributions of geometric rewards and real execution success.
- Note analysis: matching a reference 2D path does not guarantee collision-free 3D motion and may penalize another valid path. Multi-solution annotations, 3D constraints, and inference-latency evaluation are useful next steps.
Related Work & Insights¶
- vs ManipLVM-R1: both improve embodied manipulation through verifiable rewards. VLA-R1 emphasizes geometrically grounded CoT synthesis and jointly checks regions, local trajectory geometry, and formatting; Table 2 directly compares their 3B models.
- vs RoboBrain / NORA: these represent specialized embodied supervised learning approaches. VLA-R1 adds RL after CoT supervision, and Table 5 supports the value of this combination more directly than cross-model comparisons do.
- vs Embodied-R1: the paper compares trajectory prediction in Table 2 and five-scenario real-world performance in Table 4. These results support improvements on the evaluated tasks, not superiority across all robotic skills.
Rating¶
- Novelty: 3/5. The contribution mainly combines embodied CoT data and geometric rewards rather than introducing a new policy optimization algorithm.
- Experimental Thoroughness: 3/5. Evaluation covers in-domain, out-of-domain, simulated, and real settings, but splits, reward ablations, and statistical reliability remain insufficient.
- Writing Quality: 3/5. The method's overall logic is clear, but some equations are corrupted in the cached text and the real-world table contains inconsistent numbers.
- Value: 4/5. The work offers a practical way to connect language reasoning with geometric action supervision, although reproduction requires more implementation detail.