GeoSolver: Scaling Test-Time Reasoning in Remote Sensing with Fine-Grained Process Supervision

Conference: ECCV2026
Paper: ECCV Paper
Code: https://github.com/minglangL/GeoSolver
Area: Remote Sensing / Vision-Language Reasoning
Keywords: process reward model, visual faithfulness, tree-based reinforcement learning, test-time compute, remote sensing question answering

TL;DR

GeoSolver trains the token-level GeoPRM through entropy-guided tree search and visual hallucination injection, then integrates it into Process-Aware Tree-GRPO and test-time search to constrain intermediate visual evidence as well as final answers; in the paper's Table 3, visual grounding improves from 58.19 to 68.04 with a generation budget of 32.

Background & Motivation

Remote sensing vision-language models face densely arranged objects with similar appearances and widely varying scales, rather than a few prominent objects in ordinary photographs. Before answering how many aircraft are present, a model needs to locate candidates, check their positions, and exclude duplicates or mistaken detections. Models such as GeoChat and VHM already support domain-specific question answering, but a correct number does not establish that the model found the correct objects. RS-EoT and GeoZero introduce chain-of-thought reasoning that collects visual evidence before answering; this shifts the question from whether an explanation exists to whether it is faithful. If a model invents coordinates but happens to produce the correct count, answer-only training can still reward that faulty trajectory.

This blind spot of outcome rewards becomes a credit assignment problem in reinforcement learning: an entire trajectory receives a score without identifying where it departed from the image. Adding process rewards directly is not necessarily sufficient, because longer reasoning is more likely to contain low-scoring passages and the model may shorten its response to avoid penalties. General mathematical verifiers can check symbolic derivations but may struggle with shifted bounding boxes, incorrect object attributes, and dense spatial relationships in remote sensing. The paper therefore addresses supervision data, reward formulation, and exploration efficiency together, rather than merely adding a scorer to an existing model.

GeoSolver connects these problems by mining reasoning disagreements through model search, supplementing visual mismatches with synthetic perturbations, and using verification to influence both training and test-time path selection. The resulting GeoSolver-9B is the answering policy, whereas GeoPRM is a separate verifier of candidate reasoning; their responsibilities differ. The verifier can also serve other generators, so domain knowledge need not be introduced exclusively through additional generator fine-tuning. Core Idea: learn a process verifier that localizes visual reasoning errors, then use confidence-drop penalties and tree-based credit assignment to spend additional compute on visually supported reasoning paths.

Method

Overall Architecture

The input is a remote sensing image and a natural-language question; outputs can include counts, bounding boxes, scene classes, answers, or image captions. The policy starts from GLM-4.1V-9B-Base and learns a reasoning format involving planning, visual evidence collection, and synthesis through Geo-CoT380k. Its Aimv2-Huge visual encoder supports variable resolutions and aspect ratios, while the language component uses 3D-RoPE; these are inherited backbone capabilities, not a new visual architecture introduced here. The added mechanisms are Dual-Source Process Data, Token-Level GeoPRM, Process-Aware Tree-GRPO, and GeoPRM-Guided Search, in that order. During training, ground-truth answers provide annotation signals and outcome rewards; at test time, they are unavailable and the verifier judges generated content instead.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    DATA["Geo-CoT380k<br/>Images, questions, annotations"] -->|SFT and offline construction| BUILD["Dual-Source<br/>Process Data"]
    BUILD --> VERIFY["Token-Level GeoPRM"]
    VERIFY -->|Training verification feedback| TRAIN["Process-Aware<br/>Tree-GRPO"]
    DATA -->|SFT policy and outcome supervision| TRAIN
    TRAIN -->|GeoSolver-9B| SEARCH["GeoPRM-Guided Search"]
    VERIFY -->|Test-time scoring| SEARCH
    QUERY["Test image and question"] --> SEARCH
    SEARCH --> ANSWER["Reasoning path and final answer"]

Training and testing share the verifier, but this does not mean that reinforcement learning is repeated for every answer. The trained GeoSolver can use greedy decoding directly; candidate generation and verification costs are added only when test-time scaling is enabled. GeoPRM can also score trajectories from GLM-4.1V or Qwen3-VL without requiring those generators to undergo the same Tree-GRPO alignment.

Key Designs

1. Dual-Source Process Data: cover natural reasoning errors and visual mismatches

The SFT policy first generates alternative reasoning trajectories for each problem, branching at positions with high entropy in the next-token distribution. High entropy indicates uncertainty about the next choice given a prefix, rather than a fixed entropy attached to an already generated token. The algorithm selects such positions, performs multiple rollouts, and iteratively expands the reasoning tree to focus sampling on decisions that may change the outcome. For a reasoning step, its Monte Carlo value is estimated from the fraction of continuations that reach the correct answer. This label measures the current policy's success potential from that prefix; it is not equivalent to human verification that the step is visually correct. The initial pool contains 3.72 million trajectories; filtering low-variance problems by the standard deviation of correctness scores within each problem retains 1.37 million samples. Problems whose paths are all correct or all incorrect offer little discrimination between neighboring decisions and are therefore filtered.

Success-rate labels can still miss incorrect reasoning that leads to a correct answer, so a second source directly modifies visual facts in ground-truth trajectories. Bounding-box perturbations include small jitter and large displacement, targeting fine boundary errors and obvious mismatches such as boxes moved onto background regions. Fact modification changes object counts or attributes so that statements contradict the original image. These negatives and unmodified positive anchors contribute approximately 0.7 million samples, forming Geo-PRM-2M together with the search-derived data. The paper reports approximate sizes, so the dataset name should not be treated as an exact deduplicated sample count. The two sources are complementary: search supplies natural reasoning mistakes, while synthetic perturbations target visual hallucinations that final-answer rewards can miss. The main text does not fully specify how MC values become binary token labels or all perturbation magnitudes and filtering thresholds; reproduction still requires implementation details.

2. Token-Level GeoPRM: provide feedback when a local coordinate or fact becomes incorrect

GeoPRM initializes from the SFT model and adds a linear binary classification head, rather than training an answer-only scoring network from scratch. It receives the image, question, and preceding reasoning context, and predicts correctness probabilities for reasoning tokens. Consequently, a problematic coordinate number or attribute word can receive more localized feedback than a single label assigned to an entire sentence. Training uses masked token-level binary cross-entropy, computing loss only over valid reasoning tokens. Here, a correctness probability is a learned discriminative score; the paper does not provide sufficient calibration experiments to establish that it equals the true probability of error.

The value of this granularity is to move from judging an explanation as generally unreliable to identifying where a newly introduced visual fact loses support. Nevertheless, the verifier still relies on its own visual representation, rather than an independent system with geometric ground truth. It is also not a replacement for an object detector: its immediate output scores candidate text, while the policy generates boxes and answers. This separation allows one GeoPRM to inspect different generators, although cross-model capability must be bounded by the tested settings.

3. Process-Aware Tree-GRPO: adjust leaf rewards before propagating decision values

Instead of replacing answer rewards with a sum or mean of all process scores, the method checks whether confidence suddenly decreases between neighboring positions. If a drop exceeds a threshold, the trajectory's outcome score is multiplied by a penalty factor between 0 and 1. The outcome score can be continuous and task-specific; the text gives IoU and mAP as examples and places it in \([0,1]\). Thus, even a correct final answer cannot receive its unmodified reward when the trajectory contains a substantial process error. Conversely, a long but consistently credible trajectory need not accumulate penalties simply for containing more tokens; this is the design rationale, not proof that every length bias disappears.

Training rollouts also branch at high-entropy positions rather than generating every candidate chain independently from the beginning. Each complete leaf trajectory receives the adjusted reward, and an intermediate node takes the mean reward of all complete descendant trajectories as its value. Global advantage compares the node with the root, measuring improvement over the average performance of the problem's tree. Local advantage compares the node with its parent, measuring the change introduced by that decision relative to its existing prefix. The combined advantage is downweighted by the square root of the number of descendant leaves, preventing shared prefixes from being amplified through repeated appearances in complete trajectories. A clipped probability-ratio objective then updates the policy, allowing process verification to influence intermediate decisions rather than only post-hoc answer selection. The combination operator in Equation 7 is damaged in the cached extraction, so this note does not guess the exact advantage expression or the complete clipped loss.

4. GeoPRM-Guided Search: convert test-time budgets into verified candidate selection

Best-of-N generates multiple complete candidates and selects an answer using GeoPRM's process feedback; this increases the chance of finding a correct trajectory but requires reliable candidate ranking. Beam Search evaluates branches during incremental generation, allowing early removal of paths that introduce unsupported visual facts. Self-Consistency instead applies majority voting across answers without explicitly checking the visual evidence supporting each trajectory. If a generator repeatedly hallucinates in similar ways, majority voting may continue to favor the same wrong answer. GeoPRM's additional value is therefore not more generated text itself, but a discriminative signal distinct from generation probability or voting frequency.

Table 3 uses a generation budget of 32, while Figure 3 examines gains and saturation as the budget increases. The main text does not fully specify how token scores are aggregated into trajectory scores, all beam parameters, or how verification overhead enters the budget. Equal generation budgets should therefore not automatically be interpreted as equal end-to-end latency or GPU cost. Cross-model experiments use \(N\in\{8,16,32\}\) to test a fixed verifier with different generators, not to retrain on the test questions.

A Worked Example

Figure 5 on page 14 provides two aircraft-counting trajectories that illustrate where verification intervenes. The incorrect trajectory identifies several aircraft and then claims an additional aircraft at \([223,789]\) in the lower-left region during Step 5, ultimately answering 5. The authors identify that object as a hallucination and report a GeoPRM score drop from 0.966 to 0.228 around this point. The alternative trajectory counts the visible objects and answers 4 without introducing the nonexistent aircraft. During training, such a drop can reduce the erroneous trajectory's leaf reward and consequently change the advantages along its path. During test-time search, the same feedback supports candidate ranking or pruning, without using the ground-truth answer as an online selection criterion. The figure illustrates a localized error, but does not report detection recall or false-positive rates across all hallucination types.

Loss & Training

Using four NVIDIA H200 GPUs, the policy undergoes 1 epoch of SFT followed by 1000 Process-Aware Tree-GRPO optimization steps. GeoPRM trains on Geo-PRM-2M for 2 epochs with a batch size of 128; policy RL uses an expanded training set of Geo-CoT380k. SFT minimizes the negative log-likelihood of target reasoning text, GeoPRM learns masked binary classification, and RL optimizes task rewards modified by process feedback. These objectives serve format initialization, verification learning, and policy alignment respectively, rather than constituting a single end-to-end joint training objective. Only two relations supported by the prose are restated below; damaged equations are not reconstructed and presented as exact author formulas.

\[ V(s)=\frac{\text{number of successful rollouts from }s}{T} \]

Here, \(T\) is the number of continuations, and success depends on the final answer satisfying the annotation; this restates the textual definition of Equation 2 on page 6. For RL node values, the quantity averaged is instead the process-adjusted reward of complete descendant trajectories, not merely discrete answer success.

\[ GA(s)=V(s)-V(\mathrm{root}),\qquad LA(s)=V(s)-V(p(s)) \]

Here, \(p(s)\) denotes the parent node; these relations are explicitly stated in the prose on page 8. The drop threshold, penalty factor values, and full advantage-combination and loss implementations require verification against legible equations or code. The paper presents its code link as a forthcoming release; this note has not verified current repository availability.

Key Experimental Results

Main Results

The paper evaluates 6 task categories across 17 benchmarks; the following selection highlights both spatial-evidence gains and counterexamples without treating different metrics as one accuracy measure. The first table selects results from Table 1 on page 9 under standard inference without test-time search; values retain the paper's percentage scale.

Task and dataset	Metric	GeoSolver	Comparison model	Comparison score
Visual grounding, DIOR-RSVG	mIoU	75.62	GLM-4.1V-Thinking	39.41
Visual grounding, RRSIS-D	mIoU	76.66	VHM	55.20
Object detection, HRRSD	mAP@50	94.74	GLM-4.1V-Thinking	55.53
Object counting, NWPU-VHR	Accuracy	79.0	GLM-4.1V-Thinking	62.5
Object counting, RSOD	Accuracy	45.5	Gemini-2.0-Flash	63.5

HRRSD detection exceeds the listed GLM baseline by 39.21 percentage points, but RSOD counting trails Gemini, ruling out a claim of universal superiority. Table 2 on page 10 also reports 98.33 for AID scene classification and 80.93 BLEU-4 for NWPU-Captions. In that table, SIRI-WHU classification is 76.00 versus RS-EoT's 78.88, while RSICD captioning is 36.18 versus SkySenseGPT's 42.47.

The second table reproduces Table 3 on page 11, where each task column aggregates its corresponding datasets and search methods use a generation budget of 32. VG uses mIoU, Detect uses the detection metric, OC, SC, and VQA use Accuracy, and IC uses BLEU-4; the table's independent reporting convention is preserved.

Inference strategy	VG	Detect	OC	SC	VQA	IC
Greedy decoding, no TTS	58.19	74.11	54.06	90.62	70.07	47.00
Self-Consistency	59.44	75.17	59.28	92.57	76.58	47.51
GeoPRM Best-of-N	66.35	82.51	73.92	96.01	88.70	48.18
GeoPRM Beam Search	68.04	84.66	75.45	98.39	87.84	48.05

Beam Search improves VG over greedy decoding by 9.85 percentage points, but slightly trails Best-of-N on VQA and IC; neither search strategy dominates every column. Captioning gains are much smaller than counting gains, suggesting that checkable local spatial facts may be more amenable to this verification signal than free-form descriptions. This is an interpretation of the table, not a separately established causal finding.

Ablation Study

The third table selects results from Table 5 on page 13 and compares training alignment strategies; these values are kept separate from the standard-policy aggregate row in Table 3. Avg is an arithmetic composite across six different task metrics, not a unified probability of success.

Config	VG, mIoU	OC, Accuracy	Detection, mAP	Avg
SFT only	53.77	49.73	59.42	60.34
Vanilla GRPO	59.31	55.56	68.89	67.09
Add Average Process Score, APS	48.69	48.22	50.76	58.13
Add Process-Aware reward, PA	60.12	59.21	69.90	68.16
Add tree exploration only	59.46	57.77	70.82	68.13
Process-Aware Tree-GRPO	61.07	63.6	73.97	70.51

Relative to Vanilla GRPO, the full method raises Avg by 3.42, whereas APS reduces it by 8.96; the authors attribute the latter to reward hacking through truncated reasoning. Table 6 on page 13 further tests data composition: GeoSolver's object-level BoN composite is 74.26 at \(N=32\). Removing MCTS data reduces it to 64.27, while removing hallucination injection yields 70.26, differences of 9.99 and 4.00 respectively. This supports complementarity between the data sources, but does not separate changes in data quantity from changes in data type.

Key Findings

• Table 4 reports an Avg of 75.95 for GeoPRM with BoN at \(N=32\), compared with 68.43 for Self-Consistency and 62.37 for GraphPRM.
• Figure 4 shows generalist models with GeoPRM exceeding selected domain experts in the illustrated counting and scene-classification settings, not all experts on every task.
• Not all no-TTS aggregates in Table 3 can be reproduced directly from corresponding dataset means in Tables 1 and 2, and the full-model aggregates in Table 5 differ as well; the text does not clearly explain these setting differences, so this note does not reconcile or correct them.

Highlights & Insights

• The two supervision sources address different failures: search identifies natural decision disagreements, while visual perturbations expose incorrect facts missed by outcome rewards. This complementarity is more targeted than simply adding correct chains of thought.
• How rewards are used matters as much as the verifier itself. APS degradation shows that having process scores does not make directly optimizing them effective, because reward formulation changes the shortcuts a policy can exploit.
• A verifier can serve as a separate domain-adaptation component. Candidate selection can exploit remote sensing knowledge while leaving the generator unchanged, although the benefit entails additional inference cost.

Limitations & Future Work

• The paper has no dedicated systematic limitations section; the reproducibility and evaluation boundaries here are reader analysis, not all explicit author admissions.
• MC labels remain tied to the base policy and outcome evaluation, while synthetic perturbations cannot cover every natural hallucination; a human-verified benchmark of stepwise visual errors would be valuable.
• Confidence drops may miss persistently low confidence, gradual deterioration, or confidently stable errors. The mechanism avoids some length bias but cannot guarantee detection of every unfaithful path.
• Figure 3 demonstrates gains and saturation across budgets, not by itself a rigorous compute-optimal law; normalized latency, verifier FLOPs, and memory-cost comparisons are missing.
• Several cached equations are damaged, and label mapping, search-score aggregation, and some hyperparameters are underspecified; differences between aggregate tables further constrain exact reproduction.

Related Work & Insights

• Compared with GeoChat and VHM: these provide remote sensing understanding and domain adaptation; GeoSolver emphasizes explicit verification of visual facts within reasoning, not just final task scores.
• Compared with GeoZero and RS-EoT: these advance geospatial chain-of-thought reasoning; GeoSolver additionally asks where a chain becomes unfaithful and brings verification into policy optimization and test-time search.
• Compared with URSA and TreeRL: GeoSolver builds on drop-based process penalties and entropy-guided tree exploration, combining them with remote sensing mismatch data rather than introducing every component idea from scratch.
• Research direction: under a fixed total generation-and-verification cost, compare static search against budgets allocated adaptively by local uncertainty, and use real stepwise error annotations to assess pruning correctness. This is a proposal, not a completed experiment.

Rating

• Novelty: 4/5. The main contribution is the combination of remote sensing process data, training, and search; several underlying ideas come from earlier work.
• Experimental Thoroughness: 4/5. Multiple tasks, verifier comparisons, and data ablations are covered, but cost normalization and stepwise error evaluation remain limited.
• Writing Quality: 3/5. The narrative is clear, but implementation details, cross-table reporting conventions, and cached equation readability impede verification.
• Value: 4/5. It offers a transferable verifier approach for remote sensing vision-language reasoning while leaving concrete reproducibility and efficiency questions.