Semantic-Geometric Dual Compression: Training-Free Visual Token Reduction for Ultra-High-Resolution Remote Sensing Understanding

Conference: ECCV2026
Paper: ECCV paper
Code: https://github.com/Cynthia314/DualComp
Area: VLM Efficiency
Keywords: visual token compression, ultra-high-resolution remote sensing, semantic-geometric duality, task routing, topology completion

TL;DR

DualComp dynamically combines local semantic aggregation with structural path completion according to a question's object-semantic and scene-geometric demands, raising reported average accuracy from 51.5% to 53.1% while reducing inference time from 8.15 to 3.87 seconds/image in the GeoLLaVA-8K experiment on XLRS-Bench; its compression modules are training-free, but its offline router is not.

Background & Motivation

Ultra-high-resolution remote sensing imagery lets a vision-language model (VLM) inspect both large geographic regions and small objects, but preserving those details requires numerous crops and visual tokens. AnyRes-style multi-cropping turns pixel coverage into a long sequence, after which a large language model (LLM) pays the attention and generation costs of this visual context. Compression therefore needs to preserve the evidence required to answer a question, not merely a representation that broadly resembles the original scene. VisionZip, FastV, and SparseVLM provide merging or pruning mechanisms, but generic importance scores do not necessarily match the evidence demands of remote sensing tasks.

Background in the same image can play opposite roles: homogeneous ground is often redundant when counting vehicles, whereas inconspicuous regions along a road may determine whether a route is connected. Keeping only salient objects might reveal buildings at both endpoints while hiding the road between them. Conversely, retaining complete background for every task wastes a budget that object counting does not require. The pilot study in Figure 1 distinguishes semantic-dominant, balanced, and geometric-dominant responses: stronger compression can benefit the first, the second favors moderate compression, and the third is vulnerable to degradation.

The paper treats compression as question-conditioned evidence allocation rather than a fixed downsampling strength for an entire image. One branch reduces homogeneous background while retaining instance cues; another preserves boundaries, connections, and spatial skeletons, with a controller setting both the total budget and its division. This does not imply that geometric tasks need no objects, nor does it force every question into an exclusive category: continuous control variables express a preference. Core Idea: let the instruction determine how much evidence to retain and which kind, using semantic aggregation for redundant background and geometric completion for indispensable spatial connections.

Method

Overall Architecture

The inputs are an ultra-high-resolution remote sensing image and a user instruction; the host's visual encoding supplies spatially located features, while text embeddings supply the task representation. Duality-Aware Routing predicts a semantic-geometric preference and retention fraction, then allocates budgets to Spatially-Contiguous Semantic Aggregation, SCSA, and Instruction-Guided Structure Recovery, IGSR. The two branches construct semantic and geometric tokens in parallel, followed by Dual-Stream Fusion and Topological Unrolling to produce a compact visual sequence for the host model. The recovered content is structural evidence on a feature grid, not a reconstructed high-resolution pixel image or the output of an additionally trained segmentation network.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    Input["Remote sensing image + instruction"] --> Features["Visual feature grid<br/>and text embeddings"]
    Features --> Router["Duality-Aware Routing"]
    Router -->|Semantic budget| Semantic["Spatially-Contiguous<br/>Semantic Aggregation"]
    Router -->|Geometric budget| Geometry["Instruction-Guided<br/>Structure Recovery"]
    Semantic --> Fusion["Dual-Stream Fusion and<br/>Topological Unrolling"]
    Geometry --> Fusion
    Fusion --> Output["Frozen host model<br/>generates an answer"]

The two execution branches do not mean selecting semantic tokens first and then choosing geometric points from that output. They serve different evidence-preservation roles under shared routing signals, avoiding an assumption that background discarded by the semantic branch must be useless. Compression precedes language generation, reducing the visual sequence entering that stage rather than merely dropping tokens after the LLM has already processed a large visual input. Host weights remain unchanged, and the only new learnable component is the lightweight router trained offline.

Key Designs

1. Duality-Aware Routing: determine both the total budget and evidence type

The router attaches to the host's text embedding output, compresses instruction features into a compact representation, and applies a shared multilayer perceptron with two independent Sigmoid heads. It contains approximately 1M parameters and outputs a duality factor \(\lambda\) and a retention fraction \(\rho\). A smaller \(\lambda\) favors object semantics; a larger \(\lambda\) favors spatial geometry, while a larger \(\rho\) retains more tokens overall. Although the paper calls \(\rho\) a compression ratio, its budget definition makes it a retention fraction, which should not be conflated with the \(42.4\times\) compression factor in the experiments.

\[ n_{\mathrm{keep}}=N_{\max}\rho,\qquad n_{\mathrm{sem}}=n_{\mathrm{keep}}(1-\lambda),\qquad n_{\mathrm{geo}}=n_{\mathrm{keep}}\lambda. \]

Here, \(N_{\max}\) is the initial upper bound on visual tokens, and the branches receive different portions of a shared total budget. This is more specific than changing compression strength alone: the same number of tokens can represent object summaries or connecting paths. The router does not query another large model online to classify each task; Likert-scale judgments from the host LLM and expert linguistic rules are used only to generate offline preference supervision. The authors combine these preference labels using a weight \(\alpha\) and freeze the trained router, so deployment neither updates the host model nor requires supervised learning for each question.

2. Spatially-Contiguous Semantic Aggregation: merge background without averaging away small objects

SCSA uses cosine similarity between frozen CLIP visual features to find mergeable tokens in local neighborhoods rather than treating all similar-looking regions across the image as one cluster. The parent rule considers only earlier-indexed neighbors, creating a merge relation when similarity exceeds a threshold and otherwise keeping the token as its own starting point. Locality makes clusters spatially contiguous and avoids simply mixing distant, similar-looking but distinct objects. The threshold \(\tau(\lambda)\) increases monotonically with \(\lambda\): stronger semantic preference permits more aggressive aggregation, whereas stronger geometric preference preserves finer local granularity.

After clustering, SCSA uses CLS-to-patch attention from a pretrained ViT as its semantic relevance signal. A cluster's score is the sum of attention values for its constituent tokens, and the highest-scoring clusters are selected within the semantic budget. Representation is not uniform averaging: a small cluster retains its original token with the highest CLS attention, while a large cluster becomes an attention-weighted average summary. The size threshold \(\theta_{\mathrm{size}}(\lambda)\) is also controlled by routing preference, but the supplied main text does not specify its exact mapping. Retaining an original feature in a small cluster reduces the risk of diluting a small object with background; it does not guarantee detecting or preserving every small object. Cumulative attention remains a heuristic importance estimate, and cluster size is not an annotated measurement of physical object size.

3. Instruction-Guided Structure Recovery: complete connected paths between sparse anchors

IGSR compares each feature with its \(3\times3\) local average, using local variation to identify edges and structural transitions. It retains high-scoring tokens within individual subregions as structural anchors rather than selecting only global maxima, encouraging spatial coverage. Text-Aware Structural Modulation, TASM, then adjusts structural scores using CLIP text-vision similarity, biasing completion toward roads, boundaries, or other structures relevant to the question. The paper states that both geometric and textual scores are normalized, with recovery reverting toward geometry when textual evidence is weak. However, operators are missing from the corresponding equations in the extracted text, so this note preserves the verifiable mechanism without presenting a reconstructed expression as the authors' exact formula.

Anchors are not merely joined by straight lines, nor are only isolated high-scoring points retained: parallel greedy path tracing selects intermediate tokens. At each step, a candidate must lie in the current point's neighborhood and strictly decrease the Chebyshev distance to the destination anchor. Chebyshev distance is the larger of the absolute row and column differences between two grid points; the constraint forces continued progress toward the destination. Among eligible candidates, the algorithm selects the highest structural score, favoring regions with local structure or instruction relevance. Tracing for different anchor pairs uses parallel tensor operations to reduce serial path-search overhead. This is a feature-space connectivity heuristic, not a global shortest-path solver or a guarantee of recovering all real road topology when detours are necessary.

4. Dual-Stream Fusion and Topological Unrolling: encode task intent in evidence proportions and sequence order

SCSA produces condensed semantic features \(T_{\mathrm{sem}}\), while IGSR supplies geometric features \(T_{\mathrm{geo}}\) organized along connections. The authors scale each stream and concatenate them without introducing projection or normalization layers. Thus, \(\lambda\) affects both token allocation and feature magnitude, further soft-suppressing the non-dominant stream. This accommodates a frozen host, although whether such scaling is equally stable across hosts requires broader architectural validation.

\[ T_{\mathrm{fused}}=\operatorname{Concat}\!\left((1-\lambda)T_{\mathrm{sem}},\lambda T_{\mathrm{geo}}\right). \]

Geometric tokens are unrolled into a one-dimensional sequence according to path connectivity instead of returning to their original image-index order. Structures that were continuous in two dimensions can consequently remain locally adjacent in the language context, allowing native relative positional encoding, such as RoPE, to exploit that adjacency. No positional encoding parameters change, and full two-dimensional geometry is not preserved: useful connections are represented through sequence organization. The Index-Reorder ablation retains selected path tokens but restores original spatial-index ordering, separating which evidence is selected from how it is arranged.

A Worked Example

Consider a remote sensing image containing a parking lot and a road network; this is an explanatory example, not a reported single-sample result. For “How many vehicles are in the parking lot?”, the router can favor the semantic branch, allowing homogeneous ground to become large-cluster summaries while small clusters retain distinct original features. The host receives object evidence with less background interference instead of reading every ground token. For “Is there a traversable route between the two regions?”, more budget can instead go to the geometric branch. IGSR locates anchors through local variation, modulates scores by question relevance, and fills paths with structural tokens that progressively approach the destination. These tokens enter the host in connectivity order, exposing intermediate evidence rather than endpoints alone; routing values are predicted, and no fixed numerical allocation is invented here.

Loss & Training

SCSA and IGSR require no gradient training, and the host MLLM's parameters remain unchanged; the router nevertheless requires offline labels and pretraining. The title's training-free description therefore concerns compression and deployment, not a system with no training cost whatsoever. The supplied full text ends after the references and does not include the clustering and anchor-selection appendix mentioned in the main text. It also does not fully specify supervision for \(\rho\), router losses, training splits, \(\alpha\), or numerical dynamic thresholds; a training recipe cannot be inferred merely from the two-head MLP architecture.

Key Experimental Results

Main Results

The primary platform is XLRS-Bench, with 13 perception and reasoning subtasks; Avg. is the authors' reported macro-average subtask accuracy, not overall image-level accuracy. The following rows are selected from the GeoLLaVA-8K series in Table 1, page 11; all accuracies are percentages, with RC denoting Regional Counting, OLUC Overall Land Use Classification, and RP Route Planning.

Method	Table 1 compression factor	Avg.	RC	OLUC	RP
GeoLLaVA-8K	Not listed	51.5	38.0	49.0	66.0
+ VisionZip	24×	42.8	39.0	37.0	62.0
+ FastV	24×	47.1	41.0	46.0	65.0
+ SparseVLM	24×	41.2	39.0	38.0	62.0
+ DualComp	42.4×	53.1	45.0	53.0	72.0

According to Table 1, DualComp improves Avg. by 1.6 percentage points, RC by 7.0 points, and RP by 6.0 points over GeoLLaVA-8K. Not every task improves: Regional Counting with Change Detection falls from 50.0% to 48.3%, and Counting with Complex Reasoning from 52.0% to 49.0%. Table 1 leaves the GeoLLaVA-8K compression factor unspecified, whereas Table 2 lists this baseline at 24×; it should not be described as completely uncompressed.

The efficiency values below come directly from Table 2, page 11, under the GeoLLaVA-8K setup; computational cost is LLM TFLOPs, not total system FLOPs.

Metric	GeoLLaVA-8K	+ DualComp
Compression factor	24×	42.4×
Tokens per grid	24	14.2
Average visual tokens	13.8k	6.4k
LLM computation (TFLOPs)	198.1	99.8
Inference time (seconds/image)	8.15	3.87
Visual encoding and compression (seconds/image)	4.28	1.52
LLM generation (seconds/image)	3.87	2.25

The tabulated times imply approximately 2.11-fold overall acceleration; this is calculated in this note, not a speed guaranteed on different hardware. The prose on page 12 gives baseline visual tokens as 14.0k, whereas Table 2 gives 13.8k; the table value is preserved here rather than silently harmonizing them. General-backbone validation appears in Table 4, page 14: Qwen2.5-VL-7B improves from 47.4% to 47.9% Avg., with DualComp at 10.24× compression. VisionZip-10× scores 46.9% in that table; the compression factors are close, not identical, and Qwen's RC drops from 40.0% to 36.0%.

Ablation Study

The following is selected from Table 3, page 12, using the GeoLLaVA-8K evaluation on XLRS-Bench; Avg., RP, and OLUC are accuracies in percent. The table's one-decimal precision is retained rather than mixing it with differently rounded or conflicting prose values.

Config	Modification	Avg.	RP	OLUC
SCSA-only	Remove geometric stream	50.3	67.0	50.0
IGSR-only	Remove semantic stream	51.1	71.0	50.0
Top-K	Retain high-scoring structural points without paths	50.5	65.0	52.0
TASM-off	Remove textual structural modulation	51.2	71.0	48.0
Index-Reorder	Reorder path tokens by original spatial index	51.5	72.0	51.0
DualComp	Full method	53.1	72.0	53.0

Key Findings

Table 3 supports stream complementarity: the full method exceeds SCSA-only by 2.8 percentage points and IGSR-only by 2.0 points. Path completion is not another name for ordinary Top-K: the table gives Top-K an RP score of 65.0%, 7.0 points below the full method. Topological unrolling improves the overall result, but the table does not show an RP improvement because Index-Reorder and the full method both score 72.0% on RP. The prose on page 13 gives Top-K's RP as 60.00% and later its Avg. as 49.96%, conflicting with Table 3's 65.0% and 50.5%. Its claim that Top-K is the weakest ablation also conflicts with SCSA-only's tabulated 50.3%; this note quotes the table while preserving these discrepancies for verification.

Highlights & Insights

• The contribution is not another universal saliency score but recognition that background redundancy depends on the question. Budget allocation consequently has an explicit task interpretation.
• IGSR moves from important points to important chains of evidence between points. This suggests a transferable compression perspective for roads, rivers, and boundaries, although transfer still needs experimental validation.
• Index-Reorder separates selection from ordering, suggesting that compact visual representations are not just sets but structured sequences for an LLM to read.

Limitations & Future Work

• The authors report possible degradation on hybrid tasks, and this note preserves negative counting results. Average gains cannot replace task-wise analysis of budget reliability.
• The principal evidence covers one remote sensing benchmark and two backbones, without repeated-run variability or sufficient hardware reproduction details. Latency advantages remain specific to the reported setup.
• Router training details, the appendix, and several equations are incomplete in the supplied material, alongside table-prose discrepancies. Reproduction requires the original appendix, implementation, and supervision splits rather than the architecture diagram alone.
• Reader assessment: greedy tracing that must monotonically reduce distance may miss roads requiring an initial detour. Curved routes, occlusion, and cross-crop connections deserve testing without assuming that feature paths equal traversable real-world paths.

Related Work & Insights

• vs GeoLLaVA-8K: it supplies the ultra-high-resolution remote sensing host and static-compression baseline; DualComp contributes dynamic evidence scheduling rather than training a larger remote sensing model.
• vs VisionZip, FastV, and SparseVLM: these methods involve visual merging or pruning inside the LLM; DualComp emphasizes task preference and spatial connections, but its empirical comparison also changes compression factors and is not a strictly equal-budget module comparison.
• vs ZoomEarth: active perception acquires local detail through repeated zooming, whereas DualComp primarily schedules single-pass compression over available visual features. They may be complementary, but the paper does not evaluate their combination.
• Source locations: pages 7–10 describe routing, SCSA, IGSR, and fusion; Tables 1–2 are on page 11, Table 3 on page 12, conflicting prose on page 13, and cross-backbone Table 4 on page 14. Official paper and project links appear above.

Rating

• Novelty: 4/5. Explicit task-conditioned separation of semantic and geometric evidence is more targeted than a single compression-importance criterion.
• Experimental Thoroughness: 3/5. Efficiency, component, and transfer experiments are included, but broader data and complete reproduction details are missing.
• Writing Quality: 3/5. The method is clearly motivated, but the training-free qualification and table-prose discrepancies require attention.
• Value: 4/5. A practical direction for high-resolution remote sensing VLM inference, without establishing a general guarantee of topological fidelity.