From Passive Observer to Active Critic: Reinforcement Learning Elicits Process Reasoning for Robotic Manipulation¶

Conference: ECCV 2026
Paper: ECCV Official Page
Area: Robotics
Keywords: task progress estimation, process reasoning, state boundary anchoring, reinforcement learning, failure detection

TL;DR¶

PRIMO R1 combines initial images, process videos, and current images with a structured chain-of-thought cold start and GRPO driven by final progress error to train a 7B robotic process evaluator, achieving average MRA 82.90 and MAE 15.52 in a four-environment comparison and 67.0% accuracy on the unseen RoboFail benchmark.

Background & Motivation¶

Long-horizon robotic manipulation requires an estimate of how much remains to be done, not merely a final success or failure label. Sparse success rewards provide limited guidance about intermediate actions, while manually designed dense rewards often require privileged environment states or task-specific rules. Vision-language models (VLMs) offer a potential alternative using natural-language goals and camera observations. However, describing a robot as folding clothing is different from determining which parts have been folded and which completion conditions remain unmet. Recognizing familiar motions alone can assign high progress to an attempt that never produces the required object-state change.

The available visual evidence introduces another challenge. A current image alone does not reveal how the scene differs from its initial state, whereas compressed video features can lose details such as gripper contact or local object geometry. Models that use successful reference demonstrations gain an additional point of comparison, but require more input when deployed on new tasks. PRIMO instead uses the language goal to define success and explicitly places initial and current images around the process video, encouraging assessment of actual state changes rather than recognition of familiar motion patterns.

The authors also train the model to explain its assessment before generating a number, rather than supervising only a progress label. Supervised fine-tuning first teaches the organization of planning, observation, and reasoning; reinforcement learning then favors generated responses with lower final numerical error. Core Idea: constrain visual evidence with state boundaries and optimize explicit process reasoning using final progress accuracy, turning a video describer into an evaluator of task completion conditions.

Method¶

Overall Architecture¶

The inputs are an initial image before execution, a process video up to the evaluation time, a current-state image, and a language instruction describing the goal. Built on Qwen2.5-VL-7B-Instruct, the model produces a chain-of-thought (CoT) and a progress value between 0 and 100, with 0 representing the initial state and 100 indicating successful completion. The trained policy is the evaluating VLM, not a robot controller that outputs joint actions.

Training combines State Boundary Anchoring, Structured Reasoning Cold Start, and Outcome Reward Optimization. The first specifies the evidence available to the model, the second initializes its reasoning behavior, and the third updates it according to numerical prediction quality. At deployment, observations are organized in the same way and passed to the trained model. Ground-truth progress labels, successful reference demonstrations, and online reward computation are not required at inference time.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    Input["Task goal + observations"] --> Anchor["State Boundary Anchoring"]
    Anchor -->|Training input| Cold["Structured Reasoning Cold Start"]
    Data["116k CoT examples"] -.->|Supervised fine-tuning| Cold
    Cold --> RL["Outcome Reward Optimization"]
    Labels["182k training examples<br/>Ground-truth progress labels"] -.->|Format and numerical rewards| RL
    RL -.->|Parameter updates| Model["Trained PRIMO R1"]
    Anchor -->|Inference input| Model
    Model --> Output["Reasoning explanation + progress"]

Key Designs¶

1. State Boundary Anchoring: provide explicit endpoints for comparing video changes

The model receives visual evidence organized as an initial image, a process video, and a current image, rather than an isolated video. The video captures action order, the boundary images preserve spatial detail for object-state comparison, and the language goal specifies which changes matter. For folding shorts, observing arm motion is insufficient: the evaluator must compare the initially unfolded garment with the folded state required by the task. Anchoring is an input-organization and prompting strategy; the main paper does not introduce a separate novel visual encoder.

The three inputs are not interchangeable. Video can distinguish an object that was picked up and put back from one that was never picked up, whereas a single image can lose that history. Boundary images provide explicit state references for interpreting the actions. Nevertheless, the ablation does not show that all three modalities are best for every task. Their combination is particularly effective on long-horizon Behavior tasks, while some two-modality configurations achieve lower MAE on AgiBot or RoboTwin. The supported conclusion is task-dependent complementarity, not universal improvement from adding images.

2. Structured Reasoning Cold Start: connect the goal, observations, and completion judgment

The post-training corpus aggregates real robotic trajectories from AgiBot and simulated trajectories from BEHAVIOR-1k and RoboTwin. The SFT stage additionally includes EgoPlan, RoboVQA, ShareRobot, STAR, NExT-QA, and Perception Test, forming the approximately 116k-example PRIMO-R1-CoT-116k dataset. The RL stage uses the approximately 182k-example PRIMO-R1-182k dataset. These rounded sizes are part of the dataset names and should not be interpreted as exact sums of the constituent sample counts. The main paper refers to Appendix A for fine-grained progress annotation and CoT synthesis; that appendix is absent from the supplied cache, so the label-generation rules cannot be independently checked here.

Generated reasoning is organized around Planning, Observation, and Reasoning: define the desired end state and plausible steps, describe observed state changes, and align those observations with the goal to assess remaining work. These are textual structures generated by one model, not three separate networks or agents. SFT establishes this reasoning and output-format starting point. With RL alone, the model must also discover how to organize its responses from reward feedback; Table 2 shows that the two-stage combination outperforms either stage alone. The authors attribute the SFT-only failure-detection regression to format overfitting, but the reported performance differences do not independently establish that causal explanation.

3. Outcome Reward Optimization: reward accurate final estimates without labeling every reasoning step

A group of candidate responses is sampled for each input, with each response containing a reasoning chain and a final progress estimate. The reward has two components. A format reward requires <think>...</think> followed by <answer>...</answer>. An accuracy reward decreases linearly with the absolute error between predicted and ground-truth progress and is bounded between 0 and 1. An exact match receives 1, while an error reaching the maximum range receives 0. The main text gives +1 as an example positive format reward and 100.0 as an example maximum error range; these examples do not constitute a complete record of the final experimental configuration.

The textual definition accompanying Equation (2) supports the following equivalent expression for the accuracy reward. This is an explanation of the mechanism, not a character-for-character restoration of the damaged formula in the cache:

\[ r_{\mathrm{acc}}=\max\left(0,1-\frac{|\hat y-y_{\mathrm{gt}}|}{R_{\max}}\right),\qquad R=r_{\mathrm{fmt}}+r_{\mathrm{acc}}. \]

GRPO constructs relative advantages using the mean and standard deviation of rewards within the sampled group. It updates the model using a clipped probability-ratio objective and a KL constraint against a reference policy, without training an additional value network. The term critic therefore has two meanings here: PRIMO is an application-level progress evaluator, but its GRPO optimizer has no separate PPO-style value critic. Rewards check the response format and final number, not whether every sentence is grounded in the images. High reward may encourage useful reasoning behavior without certifying the intermediate explanation step by step.

A Worked Example¶

Figure 6 presents an AgiBot folding-shorts example. The initial image shows an unfolded garment, while the video shows the lower hem folded upward but the upper hem not yet fully processed. The model proposes a five-step reference plan: gather the bottom hem, fold upward, gather the top hem, fold downward, and adjust the folds.

The observation section identifies the lower-hem manipulation. The reasoning section recognizes that it accounts for only part of the work and uses the unfinished upper-hem fold as evidence against complete success. PRIMO predicts 50.0%, while Qwen2.5-VL-7B and RoboBrain-7B predict 25% and 0%, respectively. These are the displayed model predictions; 50.0% should not be treated as an independently presented ground-truth label.

The point is not to mechanically divide two completed steps by five equally weighted steps: the main paper provides no general subtask-weighting rule. The example illustrates an attempt to constrain progress using goal states and visible evidence, instead of inferring near-completion merely because the robot is manipulating the object.

Loss & Training¶

Training consists of CoT supervised fine-tuning followed by outcome-reward GRPO. SFT uses examples containing reasoning chains; the RL stage does not require separate reward labels for every step in newly sampled reasoning. Thus, the absence of dense reasoning supervision applies to the RL reward design, not to the entire pipeline.

The main paper points to Appendix H for the GRPO group size, learning rate, KL coefficient, hardware, sampled frame count, and resolution. Reasoning length and inference latency are deferred to Appendix C.2. The supplied cache ends with the references and does not include these appendices, so this note does not invent their values. Equations (1) and (3), along with parts of the metric formulas, are also damaged in text extraction; an exact optimization objective is not reconstructed from those fragments.

Key Experimental Results¶

Main Results¶

PRIMO Bench separates same-task evaluation in three seen environments, unseen-task evaluation in those environments, and evaluation in an unseen real humanoid environment. Figure 3 lists 3,000 examples for each ID/OOD task group in each of the first three environments and 4,791 Real Humanoid examples collected through teleoperation of Leju KUAVO-MY.

MAE is the mean absolute difference between predicted and ground-truth progress on the 0-to-100 scale. MRA averages whether relative errors satisfy a set of thresholds, with higher values indicating better performance. The cache does not provide a fully verifiable threshold set or specify how zero ground-truth progress is handled. Reported values are retained as given: MRA is not robot task success rate and is not assumed to equal 100 minus MAE.

The following table selects results from the paper's Table 1. Average columns summarize four environments, while the Real Humanoid columns expose cross-environment performance.

Model	Average MRA ↑	Average MAE ↓	Real Humanoid MRA ↑	Real Humanoid MAE ↓
Qwen2.5-VL-7B	67.79	29.99	56.46	34.73
Qwen2.5-VL-72B	73.80	23.80	62.29	28.10
GPT-4o	79.33	20.67	74.65	25.35
Video R1 7B	66.87	29.49	53.57	31.87
ProgressLM, recommended input configuration	78.32	20.87	84.24	15.76
PRIMO R1	82.90	15.52	72.32	21.37

PRIMO exceeds Qwen2.5-VL-72B by 9.10 average MRA points, but its Real Humanoid MRA is lower than GPT-4o and ProgressLM under the latter's recommended input configuration. The paper explicitly cautions that specialized progress models use their own recommended configurations, so these cells are not strict input-controlled comparisons. The abstract's 50% error-reduction claim does not apply uniformly against every baseline.

Ablation Study¶

The next table combines the training-strategy ablation in Table 2 with the RoboFail results in Table 3. Table 2 averages seven ID/OOD groups, unlike the four-environment average in Table 1.

Training Strategy	Seven-Group Average MRA ↑	Cross-Environment MRA ↑	RoboFail Accuracy (%) ↑
Qwen2.5-VL-7B base	67.46	48.12	57.6
SFT-only	79.35	67.30	51.0
RL-only	76.72	52.12	63.0
SFT + RL	85.28	72.32	67.0

The base model's cross-environment MRA is 48.12 in Table 2 but 56.46 in Table 1. The available main text does not explain the specific reason for this discrepancy. Both are preserved under their original table contexts rather than reconciled or mixed when calculating gains. Relative to SFT-only, the full model improves seven-group average MRA by 5.93 points, cross-environment MRA by 5.02 points, and RoboFail accuracy by 16.0 percentage points.

The following selection from Table 4 focuses on input design. It is a separate input ablation; subtracting its MAE values from the final-model results in Table 1 would not isolate an input component's contribution.

Input Configuration	Average MAE ↓	Behavior MAE ↓	Average Acc@10 ↑
Current image	59.50	51.91	2.98
Initial image + current image	46.50	49.59	13.23
Video	36.61	34.85	19.25
Video + current image	30.96	27.59	27.29
Initial image + video	34.29	32.55	22.99
Initial image + video + current image	31.43	22.73	28.89

Acc@10 is the threshold-accuracy field reported in the original table, but the main text does not expand its precise decision rule. Its values are reproduced without assuming an unverified relative- or absolute-error threshold definition.

Key Findings¶

Two-stage training helps beyond fitting seen tasks: cross-environment MRA improves from 67.30 with SFT-only to 72.32. This does not establish generalization to every unseen robot platform.
Video provides important evidence: current-image-only average MAE is 59.50, compared with 36.61 for video alone. Adding both boundary images reduces Behavior MAE to 22.73, but its average MAE of 31.43 is slightly worse than 30.96 for video plus the current image.
PRIMO's 67.0% on RoboFail ties Gemini 2.0 Flash and exceeds OpenAI o1's 61.0% by 6.0 percentage points, not a 6.0% relative improvement. This is failure-detection accuracy, not manipulation success after online correction.

Highlights & Insights¶

Separate an action occurring from a goal being achieved. Initial state, process history, and current state jointly support completion-condition verification, making false completion easier to expose than assigning progress to motion clips alone.
Continuous outcome supervision can transfer to discrete judgments. Improved failure detection without RoboFail training suggests useful temporal state representations, but this transfer alone does not establish the causal contribution of intermediate reasoning chains.
Input structure and post-training complement one another. Anchoring provides clearer evidence, SFT establishes a response structure, and GRPO adjusts behavior according to numerical quality; any one component alone does not automatically reproduce the full result.

Limitations & Future Work¶

Offline evaluation is not closed-loop control. The paper demonstrates progress estimation, failure detection, and trajectory tracking. Its main text does not report control success after integrating the rewards into policy learning or end-to-end gains from failure-triggered correction.
Explanation faithfulness remains unverified. Format and final-number rewards can accept an accurate answer with an unfaithful explanation. Interventions such as masking key frames, replacing the initial state, or shuffling temporal order could test evidence dependence; this is a reader-proposed direction.
Annotation and real-time details cannot be fully checked. Missing appendices prevent verification of progress-labeling rules, training hyperparameters, and inference latency. The available evidence does not establish readiness for real-time, high-frequency robot control.
Protocols and numerical contexts must remain distinct. Different baseline input configurations, different averaging schemes in Tables 1 and 2, and the discrepancy in base-model cross-environment results limit a single-ranking interpretation. Three-modality input also does not win every MAE column.

vs ProgressLM, VLAC, and Robo-Dopamine: As characterized by this paper, these specialized methods depend on explicit reference demonstrations. PRIMO evaluates progress using a language goal and the observed trajectory, providing a more uniform deployment interface; ProgressLM's stronger Real Humanoid results and different input configuration still matter.
vs Video R1 and Cosmos-Reason1: General video reasoning or physical-common-sense training is not the same as dedicated task-progress learning. PRIMO trains on continuous progress error and performs better in the paper's failure-detection comparison, but this is not a comprehensive comparison of all video-understanding capabilities.
vs endpoint-only approaches: Boundary images support spatial comparison but can omit long-horizon execution history. The benefit of video on Behavior indicates that process supervision needs both what happened and the resulting state.
Research direction: Jointly supervise verifiable state conditions derived from the goal and numerical progress, then test reward exploitability in actual closed-loop policies. This would evaluate the practical value of an active critic more directly than simply generating longer explanations.

Rating¶

Novelty: 4/5. The combination of boundary anchoring and outcome-reward reasoning for robotic process supervision is coherent, although its main training components have established precedents.
Experimental Thoroughness: 4/5. Multiple environments, training strategies, input ablations, and zero-shot failure detection are covered; the main text lacks closed-loop control validation and leaves some cross-table values unclear.
Writing Quality: 3/5. The motivation and qualitative examples are accessible, but some causal language exceeds the direct evidence, and evaluation conventions and reproducibility details need clarification.
Value: 4/5. A useful starting point for explainable progress estimation and robotic reward modeling, with real-time deployment and explanation faithfulness requiring further validation.