Domain Generalized Stereo Matching with Uncertainty-guided Data Augmentation

Conference: AAAI2026
arXiv: 2508.01303
Code: To be confirmed
Area: 3D Vision
Keywords: Stereo Matching, Domain Generalization, Data Augmentation, Uncertainty Modeling, Feature Consistency

TL;DR

UgDA-Stereo is proposed to simulate visual styles of various unseen domains by applying batch-statistics-based Gaussian uncertainty perturbations to the channel-wise means and standard deviations of RGB images. Combined with feature consistency constraints, this plug-and-play approach significantly enhances the cross-domain generalization capability of stereo matching models.

Background & Motivation

Stereo matching is a fundamental task in computer vision, aiming to estimate dense disparity maps from rectified image pairs for 3D reconstruction. Current state-of-the-art (SOTA) deep stereo matching networks are typically trained on synthetic data (e.g., SceneFlow), but their generalization performance severely degrades in real-world scenes due to domain discrepancies in color, illumination, contrast, and texture.

There are two existing lines of work: domain adaptation (requires target domain data) and domain generalization (does not require target domain data). This paper focuses on the more challenging single-domain generalization setting—training only on synthetic data and testing on completely unseen real-world domains.

Previous domain generalization methods either require modifying network architectures (e.g., domain normalization layers in DSMNet), rely on complex learning objectives (e.g., the information-theoretic strategy in ITSA), or require additional modalities (e.g., depth prior distillation). Although effective, these methods introduce considerable complexity. This paper attempts to start from a simple yet key observation: the statistics (mean and standard deviation) of image RGB channels carry domain feature information, and perturbing these statistics reasonably can generate samples of new domains.

Core Problem

1. Stereo matching networks trained on synthetic data tend to learn domain-specific shortcut features, failing to extract meaningful semantic and structural features in unseen real domains.
2. How to systematically expand the sample distribution via data augmentation while ensuring both diversity and validity.
3. The test domain may introduce domain shifts with uncertain directions and intensities; how to model this uncertainty.

Method

Overall Architecture

UgDA-Stereo is a plug-and-play module used only during training, which can be integrated into any stereo matching network. The overall pipeline is:

1. Apply uncertainty-guided data augmentation to the original left and right images to generate stylized images.
2. Feed the augmented image pairs into the stereo matching network to obtain disparity estimations.
3. Extract features from both original and augmented images simultaneously, and apply feature consistency constraints.

Uncertainty-guided Data Augmentation (UgDA)

Core Idea: RGB channel statistics (mean \(\mu_c\) and standard deviation \(\sigma_c\)) reflect domain features. Applying perturbations to these statistics can simulate domain shifts.

Specific Steps:

1. Calculate per-image channel statistics: Compute the mean \(\mu_c(x)\) and standard deviation \(\sigma_c(x)\) of each RGB channel along the spatial dimensions for each image.

2. Calculate variations of statistics within a batch: Measure the variances \(\sigma^2_{\mu_c}\) and \(\sigma^2_{\sigma_c}\) of image statistics within a batch. These reflect the appearance variations among images in the batch and provide a reasonable range for perturbations.

3. Gaussian sampling perturbation: Assume that the perturbation on the mean follows \(\mathcal{N}(0, \sigma^2_{\mu_c})\) and that on the standard deviation follows \(\mathcal{N}(0, \sigma^2_{\sigma_c})\). New statistics are sampled via the reparameterization trick:

   • \(\mu'_c(x) = \mu_c(x) + \epsilon_{\mu_c} \cdot \sigma_{\mu_c}(x)\), where \(\epsilon_{\mu_c} \sim \mathcal{N}(0,1)\)
   • \(\sigma'_c(x) = \sigma_c(x) + \epsilon_{\sigma_c} \cdot \sigma_{\sigma_c}(x)\), where \(\epsilon_{\sigma_c} \sim \mathcal{N}(0,1)\)

4. Generate augmented images: First normalize the original pixels, and then denormalize using the new statistics:

   • \(x^*_{c,h,w} = \frac{x_{c,h,w} - \mu_c(x)}{\sigma_c(x)} \cdot \sigma'_c(x) + \mu'_c(x)\)

Key Properties: - Only global channel-wise statistics are altered, while local textures, edges, and geometric content remain intact. - The uncertainty is modeled based on batch statistics, delivering random and diverse perturbation directions and intensities. - The same augmentation operation is applied to both left and right images, ensuring left-right consistency.

Feature Consistency Constraint

Augmentation only alters style attributes (brightness, hue, contrast) of images while preserving scene structures. Therefore, a feature consistency loss is introduced to encourage the network to learn domain-invariant representations independent of shortcuts:

\[\mathcal{L}_{cons} = \|f_{feat}(x_L) - f_{feat}(x^*_L)\|_2 + \|f_{feat}(x_R) - f_{feat}(x^*_R)\|_2\]

Loss & Training

\[\mathcal{L} = \mathcal{L}_{smooth_{L_1}}(\hat{d}, d_{gt}) + \lambda \mathcal{L}_{cons}\]

where \(\lambda = 0.17\) is a hyperparameter.

Key Experimental Results

Experimental Setup

• Training set: SceneFlow (35,454 pairs)
• Test sets: KITTI 2012, KITTI 2015, Middlebury (half resolution), ETH3D
• Baseline networks: PSMNet, GwcNet, CFNet
• Training configuration: Single RTX 4090 GPU, Adam optimizer, 20 epochs, batch size=4

Main Results (D1 Error Rate)

Method	KITTI2015	KITTI2012	Middlebury	ETH3D
PSMNet	16.3%	15.1%	34.2%	23.8%
UgDA-PSMNet	5.3%	4.8%	8.5%	10.2%
GwcNet	22.7%	20.2%	37.9%	54.2%
UgDA-GwcNet	4.9%	4.2%	8.3%	5.7%
CFNet	6.0%	5.2%	15.4%	5.7%
UgDA-CFNet	5.2%	4.7%	8.2%	4.9%

Compared to the original PSMNet, UgDA-PSMNet achieves a 10.3%–25.7% reduction in D1 error rate on the four datasets; UgDA-GwcNet shows an even larger decrease of 16.0%–48.5%.

Ablation Study (D1 Error Rate)

Augmentation	\(\mathcal{L}_{cons}\)	KITTI2012 (PSMNet)	KITTI2015 (PSMNet)
✗	✗	15.1%	16.3%
✓	✗	5.8%	6.1%
✓	✓	4.8%	5.3%

Data augmentation contributes to the primary improvement, and the feature consistency loss yields an additional gain of approximately 1%.

DrivingStereo Weather Robustness

Method	Sunny	Cloudy	Rainy	Foggy	Average
PSMNet	62.5%	60.1%	60.5%	68.6%	63.9%
FT-PSMNet	4.0%	2.9%	11.5%	6.5%	6.3%
UgDA-PSMNet	4.2%	3.3%	6.5%	5.7%	4.9%

Without using target domain data, UgDA-PSMNet even outperforms the fine-tuned baseline in rainy and foggy scenes.

Highlights & Insights

1. Extremely Simple and Efficient: The entire approach operates solely at the input level without modifying the backbone network or requiring additional modalities, resulting in extremely low computational overhead.
2. Clear Theoretical Intuition: Based on the observation that RGB statistics carry domain features, modeling perturbation uncertainty with a Gaussian distribution is highly natural.
3. Universal Plug-and-Play: Consistently achieves significant improvements across three different architectures (PSMNet, GwcNet, CFNet).
4. Robust to Adverse Weather: Demonstrates stable performance under various weather conditions in DrivingStereo, outperforming the fine-tuned version without leveraging target domain data.

Limitations & Future Work

1. Global Statistics Only: Unable to simulate local domain variations (e.g., style differences in local shadows or local occlusions), which may limit performance in fine-grained regions.
2. Limitations of Gaussian Assumption: Real-world domain shifts may not strictly follow a Gaussian distribution; modeling via more complex distributions (such as Gaussian Mixture Models or flow-based models) could potentially improve the outcomes.
3. Occlusions and Non-Lambertian Surfaces: The authors note that the handling of occluded areas and non-Lambertian surfaces (e.g., transparent/reflective objects) still has room for improvement.
4. Only Trained on SceneFlow: The impact of other synthetic datasets (e.g., TartanAir) or combinations of synthetic data has not been explored.
5. Feature Consistency Constraint via L2 Only: Stronger constraints such as contrastive learning or mutual information could be considered.

Related Work & Insights

Method	Strategy	Architecture Modification	Target Domain Data Required
DSMNet	Domain normalization layers	Yes	No
GraftNet	Large-scale pretrained features	Yes	No
ITSA	Information-theoretic perturbation + gradient adversarial	No	No
HVT	Multi-level data augmentation	No	No
Masked-Stereo	Pseudo multi-task (matching + reconstruction)	Yes	No
UgDA-Stereo	Statistical perturbation + feature consistency	No	No

The biggest advantage of UgDA-Stereo lies in its simplicity—it requires no architectural modifications, no additional pre-trained models, and no target domain data, achieving SOTA-level cross-domain generalization solely through input-level statistical perturbations and an auxiliary loss.

Insights & Connections

1. Transferability of Core Domain Generalization Idea: The concept of simulating domain shifts by perturbing channel statistics is not limited to stereo matching; it can be directly transferred to other dense prediction tasks such as monocular depth estimation, optical flow estimation, and semantic segmentation.
2. Connection to AdaIN/Style Transfer: The augmentation formulation is essentially a type of inverse normalization combined with random reparameterization, sharing deep connections with AdaIN style transfer. This suggests potential opportunities to integrate richer style libraries.
3. Batch Statistics as Uncertainty Source: Utilizing statistical variations within a mini-batch to drive augmentation is an elegant and straightforward strategy, making it suitable for other scenarios that require domain diversity.

Rating

• Novelty: 3/5 (While the concept of statistical perturbation is not entirely new, combining Gaussian uncertainty modeling with feature consistency presents novelty.)
• Experimental Thoroughness: 4/5 (Comprehensive coverage with four standard benchmarks, weather robustness evaluations, ablation studies, and visualizations.)
• Writing Quality: 3/5 (Generally clear, though with minor flaws in some mathematical formatting and phrasing.)
• Value: 4/5 (Plug-and-play, simple yet effective, and highly practical for industry deployment.)

Related Papers

• [AAAI 2026] Generalized Geometry Encoding Volume for Real-time Stereo Matching
• [ICCV 2025] Diving into the Fusion of Monocular Priors for Generalized Stereo Matching
• [CVPR 2026] What Makes Good Synthetic Training Data for Zero-Shot Stereo Matching?
• [CVPR 2026] Lite Any Stereo: Efficient Zero-Shot Stereo Matching
• [CVPR 2026] PIP-Stereo: Progressive Iterations Pruner for Iterative Optimization based Stereo Matching