Free‑CD: Probabilistically Decoupled Training-Free Open-Vocabulary Change Detection with Resolution-Invariant Feature Inversion

Conference: ECCV 2026
Paper: ECCV Official
Code: https://github.com/20374230/FreeCD
Area: Segmentation
Keywords: Remote sensing change detection, open-vocabulary change detection, training-free inference, Fourier Neural Operator, Bayesian probability correction

TL;DR

Free-CD addresses the over-segmentation and boundary ambiguity caused by instance proposal networks in traditional open-vocabulary change detection by decoupling change detection into potential change probability estimation and bi-temporal open-vocabulary segmentation without task-specific training, setting new state-of-the-art benchmarks via resolution-invariant feature upsampling (RIFI-Up) and Bayesian probability correction.

Background & Motivation

Remote sensing semantic change detection (SCD) transforms bi-temporal Earth observation imagery into actionable insights by identifying the spatial locations and semantic categories of surface changes. While fully supervised models achieve competitive pixel-level precision, they are constrained by closed, predefined vocabularies. Open-Vocabulary Change Detection (OVCD) has emerged as an appealing paradigm that leverages Vision-Language Models (VLMs) like CLIP and DINO to recognize arbitrary open-world surface transitions. However, the fundamental bottleneck lies in the granularity gap: visual foundation models emphasize high-level coarse semantic abstraction, whereas change detection demands pixel-level spatial fidelity.

Contemporary OVCD frameworks such as DynamicEarth bridge this granularity gap by employing instance extraction models (e.g., SAM or APE) to enforce fine-grained object boundaries. Nevertheless, tightly coupling change localization with category recognition imposes rigid instance boundaries that perform poorly in Earth observation scenarios. Unlike natural images with clear object instances, remote sensing scenes are predominantly composed of amorphous land cover classes (such as bare soil, farmland, and grasslands) alongside continuous transitions between semantically fuzzy classes (e.g., low vegetation versus trees). Enforcing deterministic instance masks inevitably induces severe over-segmentation—fragmenting contiguous land cover into trivial pieces—or under-segmentation, while assigning rigid single labels generates spurious confidence and cascading errors.

Rather than forcing binary change masks from error-prone instance proposals, the problem should be reformulated as predicting a continuous change probability distribution that embraces boundary fuzziness while recovering fine spatial details. Core idea: decompose open-vocabulary change detection into parallel, training-free sub-tasks of potential change probability estimation and bi-temporal open-vocabulary segmentation, using a Fourier Neural Operator-based resolution-invariant upsampler (RIFI-Up) to eliminate multi-temporal feature artifacts and a Bayesian probability correction to rectify background definition discrepancies.

Method

Overall Architecture

Free-CD decouples the decision of change occurrence from semantic categorization in probabilistic space. The overall pipeline operates across three cooperative stages. First, the potential change area proposal branch extracts low-resolution bi-temporal feature maps using a frozen visual backbone, super-resolves them via RIFI-Up, and computes pixel-wise cosine similarity to yield a potential change prior probability. Second, the bi-temporal open-vocabulary segmentation branch independently segments each temporal acquisition using open-vocabulary semantic segmentation (OVSS) models, generating per-pixel class conditional probabilities over open concepts. Third, the Bayesian probability correction module reconciles discrepancies between single-temporal and bi-temporal background definitions, followed by thresholding and mask fusion to produce the final semantic change masks.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Bi-temporal Input Images"] --> B["Potential Change Area Proposal<br/>Feature Extraction + RIFI-Up"]
    A --> C["Bi-temporal Open-Vocabulary Segmentation<br/>Single-temporal OVSS Prediction"]
    B --> D["Change Prior Probability P(F)"]
    C --> E["Category Conditional Probability P(c|C)"]
    D --> F["Bayesian Probability Correction<br/>Background Calibration & Mask Fusion"]
    E --> F
    F --> G["Final Semantic Change Mask Output"]

Key Designs

1. Potential Change Area Proposal and RIFI-Up: Eliminating resolution distortion artifacts via operator inversion

Standard feature upsampling operators such as bilinear interpolation or transposed convolutions introduce severe checkerboard artifacts and high-frequency distortion when upscaling low-resolution vision transformer features. In multi-temporal feature comparison, these spatial artifacts distort legitimate feature differences into false positive change proposals. To ensure faithful feature super-resolution, RIFI-Up introduces a resolution-invariant feature inversion constraint grounded in Fourier Neural Operators (FNO). Although ViT encoders are mathematically injective, their analytical inversion is intractable. RIFI-Up parameterizes an operator-level pseudo-inverse and enforces that the learned inverse mapping reconstructs the spatial domain image regardless of the spatial discretization grid.

In practice, RIFI-Up employs stacked parameterized Joint Bilateral Upsamplers (JBU) guided by the high-resolution RGB imagery. The cosine similarity between the super-resolved bi-temporal features is mapped to a change probability via a normalized negative Sigmoid function:

\[P(F) = \Phi_{\text{sigmoid}}\bigl(-\Phi_{\text{sim}}(F_1^{\text{up}}, F_2^{\text{up}})\bigr)\]

Because FNO operates in continuous spectral space, it remains grid-independent across arbitrary spatial scales, filtering out spurious scale-induced noise and guaranteeing that feature discrepancies represent genuine surface transformations.

2. Bayesian Probability Correction: Rectifying single- vs. multi-temporal background discrepancies

In single-temporal open-vocabulary semantic segmentation, the "background" class designates pixels that do not belong to the target foreground category set. Conversely, in multi-temporal change detection, the background event represents the union of "no change occurred" and "change occurred within non-target classes." Multiplying raw single-temporal probabilities directly with change proposals introduces systematic bias. To resolve this, a Bayesian probability correction mechanism calibrates the likelihood of valid foreground change. Let \(\mathcal{V} = \mathcal{C} \setminus \{\text{"background"}\}\) denote the foreground category set, and let \(B\) denote the event that a region is predicted as background in a single-temporal image. The corrected foreground change probability \(P(F' \mid \mathcal{C})\) is defined as:

\[P(F' \mid \mathcal{C}) = P(F) \cdot \prod_{i=1}^{2} \bigl(1 - P_i(B \mid \mathcal{C})\bigr)\]

Consequently, the joint multi-temporal semantic change probability across categories \(c_i \in \mathcal{V}\) is reformulated as:

\[P(F', c_i \mid \mathcal{C}) = P(F' \mid \mathcal{C}) \cdot P(c_i \mid \mathcal{V}), \quad c_i \in \mathcal{V}\]

This formulation constrains the segmentation branch to discriminate purely among foreground categories, eliminating ambiguous background false alarms.

3. Parameter-Free Mask Fusion: Adaptive thresholding and bi-temporal semantic assignment

To avoid brittle manual threshold tuning across varied sensors and illumination conditions, Free-CD computes the binary change mask via Otsu adaptive thresholding on the calibrated probability map:

\[M_{\text{BCD}} = \Phi_{\text{threshold}}\bigl(P(F' \mid \mathcal{C})\bigr)\]

Subsequently, the binary change mask serves as a strict spatial gating operator applied to the per-pixel dominant category predicted by the argmax operator on the foreground probabilities:

\[M_{\text{SCD}}^{i} = M_{\text{BCD}} \odot \Phi_{\text{argmax}}\bigl(P_i(c_j \mid \mathcal{V})\bigr), \quad i \in \{1, 2\}\]

This parameter-free fusion requires zero backpropagation during inference, confining semantic labels exclusively to validated change regions while preserving bi-temporal geometric alignment.

Loss & Training

While Free-CD executes inference on change detection benchmarks entirely training-free, the underlying RIFI-Up upsampler undergoes a lightweight two-stage pretraining on the general remote sensing dataset MillionAID (1 epoch, batch size 4). In the first stage, the FNO network is trained to align features with downsampled images by minimizing the \(\ell_2\) reconstruction objective \(\mathcal{L}_{\text{FNO}} = \|\mathcal{G}_\theta(F) - \Phi_{\text{down}}(I)\|_2^2\). In the second stage, with FNO weights frozen, the upsampler is optimized using a composite objective balancing multi-view feature reconstruction loss \(\mathcal{L}_{\text{rec}}\) with image restoration constraint \(\mathcal{L}_{\text{img}}\):

\[\mathcal{L} = \mathcal{L}_{\text{rec}} + \gamma \mathcal{L}_{\text{img}}\]

where \(\gamma = 0.1\) is a weighting coefficient. Random geometric transformations (scaling, zooming, padding, flipping) are incorporated within \(\mathcal{L}_{\text{rec}}\) to reinforce scale robustness. During downstream change detection evaluation, all model parameters remain frozen.

Key Experimental Results

Main Results

Free-CD was evaluated against the state-of-the-art open-vocabulary change detection framework DynamicEarth across four semantic change detection benchmarks (SECOND, HiUCD, HRSCD, JL1-SCD) and two building change detection benchmarks (LEVIR-CD, WHU-CD). Metrics include semantic change score \(F_{\text{scd}}\) and mean Intersection over Union \(\text{mIoU}\) (%).

Dataset	Backbone & Identifier Configuration	Ours (Free-CD) \(F_{\text{scd}}\) / \(\text{mIoU}\)	Prev. SOTA (DynamicEarth) \(F_{\text{scd}}\) / \(\text{mIoU}\)	Gain (\(F_{\text{scd}}\) / \(\text{mIoU}\))
SECOND	DINOv3 + SegEarth-OV	23.19 / 59.84	20.50 / 58.77	+2.69 / +1.07
HiUCD	DINOv3 + SegEarth-OV	30.00 / 54.99	23.24 / 53.21	+6.76 / +1.78
HRSCD	DINOv3 + SegEarth-OV	8.37 / 57.73	3.18 / 48.38	+5.19 / +9.35
JL1-SCD	DINOv3 + SegEarth-OV	16.15 / 52.37	8.07 / 40.31	+8.08 / +12.06
LEVIR-CD (Building)	DINOv2 + SegEarth-OV	59.1 (F1) / 41.9 (IoU)	53.6 (F1) / 36.6 (IoU)	+5.5 / +5.3
WHU-CD (Building)	DINOv2 + SegEarth-OV	64.6 (F1) / 48.6 (IoU)	57.7 (F1) / 40.6 (IoU)	+6.9 / +8.0

Ablation Study

Ablations on the upsampling strategies in the potential change area proposal stage were conducted under a unified DINOv3 + SegEarth-OV architecture (\(F_{\text{scd}}\) / \(\text{mIoU}\) in %):

Upsampler Config	Feature Inversion (FI)	Resolution Invariant (RI)	SECOND	HiUCD	HRSCD	JL1-SCD
Bilinear Interpolation	✗	✗	21.11 / 56.32	28.57 / 52.25	8.23 / 57.22	15.14 / 49.64
Standard JBU	✗	✗	21.47 / 56.43	28.73 / 53.23	8.24 / 57.27	15.51 / 50.44
Transposed Conv	✗	✗	18.72 / 54.07	28.97 / 53.85	7.89 / 56.90	15.18 / 50.91
Pretrained JBU	✗	✗	21.53 / 57.91	29.56 / 54.40	8.29 / 57.36	15.77 / 51.54
Pretrained JBU + FI	✓	✗	22.18 / 58.87	29.74 / 54.54	8.18 / 57.30	15.62 / 51.43
RIFI-Up (JBU + FI + RI, Ours)	✓	✓	23.19 / 59.84	30.00 / 54.99	8.37 / 57.73	16.15 / 52.37

Key Findings

• Resolution invariance is indispensable: Without the resolution-invariant (RI) constraint, feature inversion simply overfits specific high-resolution spatial details, resulting in catastrophic artifact propagation across temporal features. Integrating the FNO spectral formulation ensures consistent performance gains across all benchmarks, notably delivering +8.08 \(F_{\text{scd}}\) on JL1-SCD.
• Superior handling of contiguous land cover: Avoiding rigid instance extraction allows Free-CD to preserve semantic continuity for large-scale rural, agricultural, and hydrological areas, where instance-based methods (e.g., SAM) suffer from severe fragmentation.
• Broad model compatibility: Free-CD integrates seamlessly across diverse vision-language models (CLIP, RemoteCLIP, GeoRSCLIP) with minimal adaptation overhead, verifying the plug-and-play generality of the probabilistic decoupling architecture.

Highlights & Insights

• Probabilistic decoupling over rigid boundaries: Formulating change detection as continuous probability estimation bypasses the fragility of instance proposal models in amorphous remote sensing scenes while preserving natural semantic transitions.
• Neural operator inversion for feature upsampling: By formulating feature inversion within continuous function space via Fourier Neural Operators, RIFI-Up achieves grid-invariant feature reconstruction that eliminates resolution-induced artifacts.
• Training-free, zero-shot adaptability: The framework operates fully training-free during evaluation, allowing rapid conversion of any pretrained open-vocabulary segmenter into an open-vocabulary change detector.

Limitations & Future Work

• Performance upper-bounded by single-temporal segmenters: When paired with segmenters weak in earth observation concepts, the framework's semantic discrimination reaches a ceiling, indicating reliance on the underlying open-vocabulary segmentation model.
• Sensitivity to severe co-registration parallax: Calculating change probabilities via pixel-wise cosine similarity assumes strict geometric alignment; building lean and sensor parallax in high-relief urban areas may introduce edge misclassifications.
• Future directions: Integrating deformable attention or unsupervised optical flow alignment into the proposal branch could further alleviate parallax-induced errors in non-nadir acquisitions.

Related Work & Insights

• vs DynamicEarth: DynamicEarth relies on sequential instance segmentation (SAM/APE) followed by classification, which leads to over-segmentation on contiguous land parcels. Free-CD employs parallel probabilistic decoupling, yielding superior boundary fidelity and category discrimination.
• vs FeatUp: FeatUp is designed for natural images without explicit resolution-invariant constraints across scales. RIFI-Up introduces FNO-based resolution-invariant inverse mapping tailored to multi-scale aerial and satellite observations.

Rating

• Novelty: ⭐⭐⭐⭐☆ Decouples change detection into continuous probability estimation with FNO-guided feature inversion.
• Experimental Thoroughness: ⭐⭐⭐⭐⭐ Comprehensive evaluation across six benchmarks with rigorous ablations and qualitative visualizations.
• Writing Quality: ⭐⭐⭐⭐⭐ Rigorous mathematical formulation of probability correction and neural operator constraints.
• Value: ⭐⭐⭐⭐⭐ Delivers an effective, training-free baseline for practical open-vocabulary remote sensing change monitoring.